Multimodality & Society最新文献_第2页

Reflections on Cope and Kalantzis: Generative Artificial Intelligence and humans 对科普和卡兰茨斯的思考：生成式人工智能与人类

Multimodality & Society Pub Date : 2024-01-27 DOI: 10.1177/26349795241230966

James Paul Gee, Qing Archer Zhang

引用次数: 0

Transforming spectator into collaborator during a digitalised dance performance 在数字化舞蹈表演中将观众变为合作者

Multimodality & Society Pub Date : 2024-01-20 DOI: 10.1177/26349795241228473

Friendred (Youhong) Peng

{"title":"Transforming spectator into collaborator during a digitalised dance performance","authors":"Friendred (Youhong) Peng","doi":"10.1177/26349795241228473","DOIUrl":"https://doi.org/10.1177/26349795241228473","url":null,"abstract":"The traditional construct of dance performances has predominantly positioned the audience as passive observers with little to no interaction with the unfolding narrative on stage. However, the advent of digital technology has paved the way for a more immersive and interactive performative experience. This paper elucidates the transformation of this dynamic through the lens of an interactive dance performance piece, “Moving Photon”, which endeavors to blur the entrenched demarcations between performers and spectators. By melding choreographic improvisation, interactive installations, and physiological sensing, “Moving Photon” cultivates a co-constitutive interaction, challenging conventional performative frameworks. Through two distinct configurations, performance and installation setting, the piece endeavors to transition spectators from passive observers to active collaborators in the narrative. The performance setting leverages real-time electroencephalogram (EEG) data from audience members to influence the kinetic dynamics of the installation, thus directly impacting the dancer's improvisational choreography. Conversely, the setup of the interactive installation invites participants to influence the musical score and the installation's robotic movements through motion-sensing devices, fostering a tangible feedback loop. The reflections gathered from initial exhibitions reveal a heightened sense of connectivity and shared authorship of the performance among both the performer and audience members. This innovative exploration signifies a step forward towards fostering more collaborative and multimodal engagements in digital art performances.","PeriodicalId":153235,"journal":{"name":"Multimodality & Society","volume":"8 46","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139523918","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Editorial multimodality and society 编辑多模式与社会

Multimodality & Society Pub Date : 2024-01-05 DOI: 10.1177/26349795231223189

Elisabetta Adami, Arlene Archer, A. Björkvall, Clarice Gualberto, Carey Jewitt, Lalitha Vasudevan, Fei Victor Lim

引用次数: 0

A multimodal grammar of artificial intelligence: Measuring the gains and losses in generative artificial intelligence 人工智能的多模式语法：衡量生成式人工智能的得失

Multimodality & Society Pub Date : 2023-12-28 DOI: 10.1177/26349795231221699

B. Cope, M. Kalantzis

{"title":"A multimodal grammar of artificial intelligence: Measuring the gains and losses in generative artificial intelligence","authors":"B. Cope, M. Kalantzis","doi":"10.1177/26349795231221699","DOIUrl":"https://doi.org/10.1177/26349795231221699","url":null,"abstract":"This paper analyzes the scope of Artificial Intelligence (AI) from the perspective of a multimodal grammar. Its focal point is Generative AI, a technology that puts so-called Large Language Models to work. The first part of the paper analyzes Generative AI, based as it is on the statistical probability of one token (a word or part of a word) following another. If the relation of tokens is meaningful, this is circumstantial and no more, because its mechanisms of statistical analysis eschew any theory of meaning. This is the case not only for the written text that Generative AI leverages, but by extension image and multimodal forms of meaning that it can generate. The AI can only work with non-textual forms of meaning after applying language labels, and to that extent is captive not only to the limits of probabilistic statistics but the limits of written language as well. While acknowledging gains arising from the brute statistical power of Generative AI, in its second part the paper goes on to map what is lost in its statistical and text-bound approaches to multimodal meaning-making. Our measure of these gains and losses is guided by the concept of grammar, defined here as a theory of the elemental patterns of meaning in the world—not just written text and speech, but also image, space, object, body, and sound. Ironically, a good deal of what is lost by Generative AI is computable. The third and final part of the paper briefly discusses educational applications of Generative AI. Given both its power and intrinsic limitations, we have been experimenting with the application of Generative AI in educational settings and the ways it might be put to pedagogical use. How does a grammatical analysis help us to identify the scope of worthwhile application? Finally, if more of human experience is computable than can be captured in text-bound AI, how might it be possible at the level of code to create a synthesis in which grammatical and multimodal approaches complement Generative AI?","PeriodicalId":153235,"journal":{"name":"Multimodality & Society","volume":"39 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139150413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Embodying intercultural communicative competence in French L2 video reflections: Illustrating skills, knowledge, and attitudes 在法语第二语言视频反思中体现跨文化交际能力：说明技能、知识和态度

Multimodality & Society Pub Date : 2023-12-21 DOI: 10.1177/26349795231222917

Julia A Gorham, Natalie A. Amgott

{"title":"Embodying intercultural communicative competence in French L2 video reflections: Illustrating skills, knowledge, and attitudes","authors":"Julia A Gorham, Natalie A. Amgott","doi":"10.1177/26349795231222917","DOIUrl":"https://doi.org/10.1177/26349795231222917","url":null,"abstract":"Research has long touted and recently confirmed the importance of intercultural communicative competence (ICC) — the ability to appropriately communicate with those from diverse sociocultural and linguistic backgrounds — in second language (L2) teaching and learning. Additionally, while multimodal course materials have contributed to learners’ ICC, the link between students’ embodied modes (gestures, facial expressions, body movements) and ICC remains under-explored. Building on ICC and the social semiotic theory of multimodality, this study blends multimodal transcription and conversation analysis to examine how 73 university undergraduate learners of L2 French used embodied modes in 188 asynchronous video reflections and how these modes accompanied demonstrations of ICC skills, knowledge, and attitudes. Findings indicate that gestures and facial expressions indicated students’ demonstrations of the ICC skills of observing, analyzing/interpreting, evaluating, relating, listening, questioning, researching, and problematizing. Further, embodied modes demonstrated students’ retrieval and communication of cultural and sociolinguistic knowledge. This study has implications for online language learning and enhancing ICC through virtual multimodal assignments.","PeriodicalId":153235,"journal":{"name":"Multimodality & Society","volume":"77 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138952870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Wood-off-cuts as a creative resource: A metaphorical reflection on “The Ruler and The Ruled” sculpture project I 作为创作资源的木材边角料：对 "统治者与被统治者 "雕塑项目的隐喻思考 I

Multimodality & Society Pub Date : 2023-12-18 DOI: 10.1177/26349795231223678

Chikelue Chris Akabuike

引用次数: 0

Book review: Digital communication and media linguistics 书评:数字传播与媒体语言学

Multimodality & Society Pub Date : 2023-08-31 DOI: 10.1177/26349795231200643

Ida Melander

引用次数: 0

Perspectival Kinaesthetic Imaging 透视运动美学成像

Multimodality & Society Pub Date : 2023-08-09 DOI: 10.1177/26349795231191023

L. Malafouris, R. Carnegie, Miranda Creswell, A. Iliopoulos, M. Koukouti, Wendy Ross

引用次数: 0

“I am feeling blue… and red”: Exploring alternative ways of feeling colours through a wearable haptic artefact “我感觉蓝色和红色”:通过可穿戴的触觉人工制品探索感受颜色的不同方式

Multimodality & Society Pub Date : 2023-08-08 DOI: 10.1177/26349795231194746

Ricardo O'Nascimento

{"title":"“I am feeling blue… and red”: Exploring alternative ways of feeling colours through a wearable haptic artefact","authors":"Ricardo O'Nascimento","doi":"10.1177/26349795231194746","DOIUrl":"https://doi.org/10.1177/26349795231194746","url":null,"abstract":"Wearable haptic artefacts are more and more present in our everyday life. They increasingly mediate our perception, transforming the ways in which we interact with ourselves, the environment and others. Wearable haptic artefacts can sense the body and its surroundings, impacting our subjectivity by shaping perceptions, actions, and behaviours. Such ability to influence our senses is helpful in creating engaging multisensory experiences. In my work, I explore the technological mediation between the human body and its surroundings by facilitating new relationships among the senses. This article presents a wearable haptic artefact in the shape of a glove called “I am feeling blue… and red”, which was designed to enable users to experience colour through touch. The goal is to suggest a mode of colour experience and break preconceptions one might have towards the relationship there is between touch and colours. Through an autoethnographic account, the article reflects on the artefact’s design process and user interaction. It proposes strategies for developing unique multisensory experiences using wearable haptic artefacts. These strategies can assist artists and designers who want to create captivating interactions by connecting the senses of sight and touch.","PeriodicalId":153235,"journal":{"name":"Multimodality & Society","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114143281","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multimodal methods for managing deontic rights in interdisciplinary trials 跨学科试验中道义权管理的多模式方法

Multimodality & Society Pub Date : 2023-07-25 DOI: 10.1177/26349795231188727

Emilie Munch Nicolaisen, Gitte Rasmussen

引用次数: 0

Multimodality &amp; Society最新文献

Multimodality & Society最新文献