Spatial working memory is critical for gesture processing: Evidence from gestures with varying semantic links to speech.

IF 3.2 3区 心理学 Q1 PSYCHOLOGY, EXPERIMENTAL
Demet Özer, Aslı Özyürek, Tilbe Göksun
{"title":"Spatial working memory is critical for gesture processing: Evidence from gestures with varying semantic links to speech.","authors":"Demet Özer, Aslı Özyürek, Tilbe Göksun","doi":"10.3758/s13423-025-02642-4","DOIUrl":null,"url":null,"abstract":"<p><p>Gestures express redundant or complementary information to speech they accompany by depicting visual and spatial features of referents. In doing so, they recruit both spatial and verbal cognitive resources that underpin the processing of visual semantic information and its integration with speech. The relation between spatial and verbal skills and gesture comprehension, where gestures may serve different roles in relation to speech is yet to be explored. This study examined the role of spatial and verbal skills in processing gestures that expressed redundant or complementary information to speech during the comprehension of spatial relations between objects. Turkish-speaking adults (N=74) watched videos describing the spatial location of objects that involved perspective-taking (left-right) or not (on-under) with speech and gesture. Gestures either conveyed redundant information to speech (e.g., saying and gesturing \"left\") or complemented the accompanying demonstrative in speech (e.g., saying \"here,\" gesturing \"left\"). We also measured participants' spatial (the Corsi block span and the mental rotation tasks) and verbal skills (the digit span task). Our results revealed nuanced interactions between these skills and spatial language comprehension, depending on the modality in which the information was expressed. One insight emerged prominently. Spatial skills, particularly spatial working memory capacity, were related to enhanced comprehension of visual semantic information conveyed through gestures especially when this information was not present in the accompanying speech. This study highlights the critical role of spatial working memory in gesture processing and underscores the importance of examining the interplay among cognitive and contextual factors to understand the complex dynamics of multimodal language.</p>","PeriodicalId":20763,"journal":{"name":"Psychonomic Bulletin & Review","volume":" ","pages":""},"PeriodicalIF":3.2000,"publicationDate":"2025-02-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Psychonomic Bulletin & Review","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.3758/s13423-025-02642-4","RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, EXPERIMENTAL","Score":null,"Total":0}
引用次数: 0

Abstract

Gestures express redundant or complementary information to speech they accompany by depicting visual and spatial features of referents. In doing so, they recruit both spatial and verbal cognitive resources that underpin the processing of visual semantic information and its integration with speech. The relation between spatial and verbal skills and gesture comprehension, where gestures may serve different roles in relation to speech is yet to be explored. This study examined the role of spatial and verbal skills in processing gestures that expressed redundant or complementary information to speech during the comprehension of spatial relations between objects. Turkish-speaking adults (N=74) watched videos describing the spatial location of objects that involved perspective-taking (left-right) or not (on-under) with speech and gesture. Gestures either conveyed redundant information to speech (e.g., saying and gesturing "left") or complemented the accompanying demonstrative in speech (e.g., saying "here," gesturing "left"). We also measured participants' spatial (the Corsi block span and the mental rotation tasks) and verbal skills (the digit span task). Our results revealed nuanced interactions between these skills and spatial language comprehension, depending on the modality in which the information was expressed. One insight emerged prominently. Spatial skills, particularly spatial working memory capacity, were related to enhanced comprehension of visual semantic information conveyed through gestures especially when this information was not present in the accompanying speech. This study highlights the critical role of spatial working memory in gesture processing and underscores the importance of examining the interplay among cognitive and contextual factors to understand the complex dynamics of multimodal language.

空间工作记忆对手势加工至关重要:来自手势与言语不同语义联系的证据。
手势通过描绘指示物的视觉和空间特征来表达对言语的冗余或补充信息。在这样做的过程中,他们需要空间和语言认知资源,这些资源支持视觉语义信息的处理及其与言语的整合。空间和语言技能与手势理解之间的关系,其中手势可能在言语方面扮演不同的角色,还有待探索。本研究考察了空间和语言技能在理解物体间空间关系过程中,对表达多余或补充信息的手势的处理中所起的作用。说土耳其语的成年人(N=74)观看了描述物体空间位置的视频,这些视频涉及到说话和手势的换位思考(左-右)或不换位思考(上-下)。手势要么向言语传达多余的信息(例如,说“左”并做手势),要么补充言语中伴随的指示(例如,说“这里”,做手势“左”)。我们还测量了参与者的空间(科西块跨度和心理旋转任务)和语言技能(数字跨度任务)。我们的研究结果揭示了这些技能和空间语言理解之间微妙的相互作用,这取决于信息表达的方式。有一个观点凸显出来。空间技能,尤其是空间工作记忆能力,与通过手势传达的视觉语义信息的理解能力增强有关,尤其是当这些信息没有出现在伴随的演讲中时。本研究强调了空间工作记忆在手势加工中的重要作用,并强调了研究认知和语境因素之间的相互作用对于理解多模态语言的复杂动态的重要性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
6.70
自引率
2.90%
发文量
165
期刊介绍: The journal provides coverage spanning a broad spectrum of topics in all areas of experimental psychology. The journal is primarily dedicated to the publication of theory and review articles and brief reports of outstanding experimental work. Areas of coverage include cognitive psychology broadly construed, including but not limited to action, perception, & attention, language, learning & memory, reasoning & decision making, and social cognition. We welcome submissions that approach these issues from a variety of perspectives such as behavioral measurements, comparative psychology, development, evolutionary psychology, genetics, neuroscience, and quantitative/computational modeling. We particularly encourage integrative research that crosses traditional content and methodological boundaries.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信