{"title":"Book review: Multimodal Literacy in School Science: Transdisciplinary Perspectives on Theory, Research and Pedagogy","authors":"Jing Song, Jinyou Zhou","doi":"10.1177/14703572231157037","DOIUrl":"https://doi.org/10.1177/14703572231157037","url":null,"abstract":"","PeriodicalId":51671,"journal":{"name":"Visual Communication","volume":"201 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73541943","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Script-switching in Japanese pop culture: a social semiotic multimodal approach","authors":"Y. Matsuda","doi":"10.1177/14703572231155586","DOIUrl":"https://doi.org/10.1177/14703572231155586","url":null,"abstract":"This article explores the nature of multimodal meaning-making in writing by analyzing the script-switching used in Japanese pop culture. Japanese intermixes multiple script systems in one text. Although official and public documents follow standard orthographic conventions, creative works in pop culture texts freely and regularly break these norms by switching script types to create new meanings. However, while many previous works recognize the semiotic effects of script-switching, to this day, there is no systematic account for how and why such meaning-making operates. Notably, script-switching triggers meanings independent of linguistic meaning. Yet, scripts are a visual manifestation of language, so one cannot say they are independent of language. Accordingly, this work attempts to resolve this dilemma by exploring a social semiotic multimodal approach (see Van Leeuwen ‘Typographic meaning’, 2005, and ‘Towards a semiotics of typography’, 2006, and Stöckl ‘Typography: Body and dress of a text’, 2005), which claims the semiotic independence of typography and its interdependency with language. By recognizing the basic unit of the writing system as graphemes, the author argues that the Japanese script systems are categorized into four types of graphemes depending on the linguistic unit they relate to and she demonstrates how the visual unit of the grapheme and the linguistic unit interact in meaning-making. In this analysis, script-switching occurs when grapheme types are intentionally changed to foreground a visual meaning associated with each script system. This article shows that such a meaning reflects the historical development and discursive social practices of writing with multiple script systems throughout the history of the Japanese writing system.","PeriodicalId":51671,"journal":{"name":"Visual Communication","volume":"39 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2023-02-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81068832","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Fictional mapping: the nature of cartography in film production","authors":"Welby Ings","doi":"10.1177/14703572231152665","DOIUrl":"https://doi.org/10.1177/14703572231152665","url":null,"abstract":"This practitioner piece considers cartographies that permeate and enable the practice of film production. Using his feature film PUNCH (2022) as a case study, Welby Ings discusses ways in which mud maps, overhead maps and relational maps were used as communicative devices in establishing a shared template for understanding the positioning of fictional spaces. Mud maps (see Danesh’s ‘7 pro tips for managing crew parking when filming on location’, 2019), are conventionally used in film production to describe geographies and inform cast and crew about locations. Overhead maps (see Isham’s ‘Creating overhead maps for film directors’, 2017), were employed in association with shot lists, as ‘graphic prompts’ when directing complex scenes. Relational or ‘mental maps’ (see Bays’, ‘Between the scenes: Choosing locations’, 2015) were used in conjunction with site drawings, to establish spatial blueprints for discussing the internal logic of the film’s world with the director of photography, production designers, location scouts, sound recordists, costumers and postproduction collaborators. As the film’s director and cartographer, the author created these maps with a unique visual language that included distinctive considerations of marginalia, typography, line weight, texture, historical allusion and colour, to communicate the emotional tone of the film’s world.","PeriodicalId":51671,"journal":{"name":"Visual Communication","volume":"73 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2023-02-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86876061","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Book review: Modelling Paralanguage Using Systemic Functional Semiotics: Theory and Application","authors":"Xiaoqin Wu","doi":"10.1177/14703572231152667","DOIUrl":"https://doi.org/10.1177/14703572231152667","url":null,"abstract":"","PeriodicalId":51671,"journal":{"name":"Visual Communication","volume":"9 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2023-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72480254","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Alien domesticity: representing home during a pandemic.","authors":"Brent Luvaas","doi":"10.1177/14703572221089012","DOIUrl":"https://doi.org/10.1177/14703572221089012","url":null,"abstract":"<p><p>Cultural geography and the related disciplines of urban sociology and anthropology have long focused their theoretical lenses on the city as a space of lived multiplicity. This photographic essay focuses its lens on the home as such a space. During the pandemic lockdown of 2020-2021, many of us spent more time in our homes than we ever had before, working, teaching, schooling, shopping, and barricading ourselves from the outside world. This essay borrows from the often ambiguous and anonymizing aesthetics of street photography to depict the multiple, overlapping worlds of home during a pandemic. Home, as depicted here, is an always unfinished process of affective assemblage and dissolution. The images featured seek to capture that lack of resolution, the messy emotional texture of home life under lockdown.</p>","PeriodicalId":51671,"journal":{"name":"Visual Communication","volume":"22 1","pages":"13-26"},"PeriodicalIF":1.5,"publicationDate":"2023-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9902811/pdf/10.1177_14703572221089012.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10694932","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Walking through the city soundscape: an audio-visual analysis of sensory experience for people with psychosis.","authors":"Sara Merlino, Lorenza Mondada, Ola Söderström","doi":"10.1177/14703572211052638","DOIUrl":"https://doi.org/10.1177/14703572211052638","url":null,"abstract":"<p><p>This article discusses how an aspect of urban environments - sound and noise - is experienced by people walking in the city; it particularly focuses on atypical populations such as people diagnosed with psychosis, who are reported to be particularly sensitive to noisy environments. Through an analysis of video-recordings of naturalistic activities in an urban context and of video-elicitations based on these recordings, the study details the way participants orient to sound and noise in naturalistic settings, and how sound and noise are reported and reexperienced during interviews. By bringing together urban context, psychosis and social interaction, this study shows that, thanks to video recordings and conversation analysis, it is possible to analyse in detail the multimodal organization of action (talk, gesture, gaze, walking bodies) and of the sensory experience(s) of aural factors, as well as the way this organization is affected by the ecology of the situation.</p>","PeriodicalId":51671,"journal":{"name":"Visual Communication","volume":"22 1","pages":"71-95"},"PeriodicalIF":1.5,"publicationDate":"2023-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/67/88/10.1177_14703572211052638.PMC9900689.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10693779","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The role of rhythm in science-animated videos: construing entities and bridging across different semiotic modes","authors":"Yufei He","doi":"10.1177/14703572221112680","DOIUrl":"https://doi.org/10.1177/14703572221112680","url":null,"abstract":"Recent decades have witnessed the increasing popularity of animation used for science education. However, the affordances of animation are still yet to be described in a comprehensive manner. Working on a corpus of online science animated videos, this article firstly proposes a visual rhythm system for animation, and then examines the convergence and divergence of the rhythm in animation and language at different levels of analysis. The author found that different visual rhythms in animation function to construe different quantities and properties of entities. In terms of the intermodal relations between animation and language, at shorter wavelengths, animation can be in sync with the rhythm in language, with different ‘beats’ in animation converging with different phonological units in language; at longer wavelengths, the larger rhythmic pattern of animation can function to scaffold the generic structure of language. This article enriches the systematic description of the meaning-making potential of animation, which also informs empirical studies focusing on the effectiveness of animation used for science education.","PeriodicalId":51671,"journal":{"name":"Visual Communication","volume":"22 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2023-01-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75055706","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"How existing literary translation fits into film adaptations: the subtitling of neologisms in Harry Potter from a multimodal perspective","authors":"Siwen Lu","doi":"10.1177/14703572221141959","DOIUrl":"https://doi.org/10.1177/14703572221141959","url":null,"abstract":"Existing literature on adaptation studies focuses primarily on analysing film adaptations from an intralingual and monomodal rather than an interlingual and multimodal perspective. To fill this gap, this study addresses the relatively under-researched issue of applying existing literary translation to the subtitles of film adaptations by the film subtitle producers. Concentrating on the Chinese subtitling of neologisms in the Harry Potter films (2001–2011) and by drawing on the Systemic Functional Linguistics (SFL)-informed multimodal framework, the aim is to investigate how subtitles and other multimodal resources interact to make meanings and their potential effects on the subtitled films when the film subtitle producers apply literary translation to subtitles of film adaptations. The results show that the application of literary translation to subtitled films by the film subtitlers may run the risk of downplaying some crucial elements of the original, such as the relationship between the fictional world and the audience. This study highlights the importance of considering more than just the literary elements when analysing film adaptations and points out broader possible areas, such as multimodality and audiovisual translation, which have only been partly recognized in adaptation studies.","PeriodicalId":51671,"journal":{"name":"Visual Communication","volume":"116 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2023-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84930369","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Visual protest repertoires and protesters’ health identity: a battlefield of the anti-new normal movement","authors":"Zhen Sun, Wei Luo","doi":"10.1177/14703572221141973","DOIUrl":"https://doi.org/10.1177/14703572221141973","url":null,"abstract":"This study explores the health identity of the anti-new normal protesters during the coronavirus pandemic through analyzing the visual protest repertoires that were employed by the protesters in the two waves of demonstrations occurring across the world between 2020 and 2021. The authors examine the protesters’ interpretations of the coronavirus and its mandatory protective measures, their imaginations of the social milieu that has been forever transformed by COVID-19, and their expectations of life. The protesters prioritize the innate physical body, which is essentialized in that there is a static relationship between the physical body and the self as well as between the self and others. The current study enriches our understandings of the new normal opponents’ health identity and provides insights into how it differs from that of the supporters. It helps health educators to effectively construct and deliver health promotion messages and foster healthy behaviors during the pandemic.","PeriodicalId":51671,"journal":{"name":"Visual Communication","volume":"45 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2022-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74966038","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ielka Van der sluis, Gabriela Matouskova, Hannah Niemeier, Sophia Popp, Josephine Carstens
{"title":"The clarity and correctness of visualized thrust actions: a description and insights from users and experts","authors":"Ielka Van der sluis, Gabriela Matouskova, Hannah Niemeier, Sophia Popp, Josephine Carstens","doi":"10.1177/14703572221135240","DOIUrl":"https://doi.org/10.1177/14703572221135240","url":null,"abstract":"This article presents three studies that evaluate the effectiveness of instructional pictures that visualize Heimlich maneuver thrusts. Firstly, a corpus study is used to describe a collection of 30 pictures employing a model in which the angle, perspective, body and hand positions of the first-aid helper and the victim, the thrust action and its depicted results are analysed. Results show a large variation in the visualizations of the same action. Secondly, 56 potential users of the pictures filled out a questionnaire to evaluate the clarity and correctness with which the action is displayed in five representative pictures from the corpus with Heimlich’s description of the thrust action as a reference. Results showed that the pictures were considered far from perfect. In particular, the clarity of the visualization of the helper applying pressure on the abdomen of the victim was considered the least clear and incorrect. Thirdly, six certified first-aid instructors gave their expert opinions on the five pictures. Their comments concentrate on the location of the hands which are depicted dangerously high on the front of the victim. These studies advance the annotation and evaluation methodologies for the investigation of visualized actions. Results serve authoring and analysis of multimodal instructions.","PeriodicalId":51671,"journal":{"name":"Visual Communication","volume":"21 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2022-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91373821","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}