Do passive cross-modal validation processes occur when processing multimedia materials?

IF 4.7 1区 教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH
Anne Schüler, Pauline Frick
{"title":"Do passive cross-modal validation processes occur when processing multimedia materials?","authors":"Anne Schüler,&nbsp;Pauline Frick","doi":"10.1016/j.learninstruc.2024.101956","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><p>In text comprehension research, a passive validation mechanism has been observed that checks the consistency between incoming and previous text information (or prior knowledge).</p></div><div><h3>Aims</h3><p>In two pre-registered online studies, we investigated whether a passive cross-modal validation mechanism occurs during the processing of multimedia materials (i.e., text combined with pictures).</p></div><div><h3>Samples</h3><p>Participants (Experiment 1: <em>N</em> = 146; Experiment 2: <em>N</em> = 235) were recruited via Prolific.</p></div><div><h3>Methods</h3><p>We used the epistemic Stroop paradigm (Richter et al., 2009), which makes use of the fact that the passive validation mechanism induces positive or negative response tendencies that can interfere with the processing of an unrelated task if it requires an opposite response. Participants received either matching (valid) or mismatching (invalid) text-picture stimuli. Following each stimulus, participants performed an unrelated probe-word task reacting to the probe words “wrong” or “right”. The dependent variables were reaction time and error rates in the unrelated probe-word task. Experiment 1 used one-sentence-picture stimuli, while Experiment 2 used longer text-segments-picture stimuli.</p></div><div><h3>Results</h3><p>Linear mixed-effects models showed interactions of validity and probe word for reaction times (Experiments 1 &amp; 2) and error rates (Experiment 1). Post-hoc comparisons indicated prolonged reaction times or higher error rates when the probe word task required a response opposite to the outcome of the validation process.</p></div><div><h3>Conclusions</h3><p>This study is the first to demonstrate that a passive cross-modal validation mechanism checks the consistency between written text and accompanying pictures. We discuss the implications of these findings for understanding information processing in multimedia contexts.</p></div>","PeriodicalId":48357,"journal":{"name":"Learning and Instruction","volume":null,"pages":null},"PeriodicalIF":4.7000,"publicationDate":"2024-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0959475224000835/pdfft?md5=80c4216addce7fcfd40de1fd5fb491d1&pid=1-s2.0-S0959475224000835-main.pdf","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Learning and Instruction","FirstCategoryId":"95","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0959475224000835","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0

Abstract

Background

In text comprehension research, a passive validation mechanism has been observed that checks the consistency between incoming and previous text information (or prior knowledge).

Aims

In two pre-registered online studies, we investigated whether a passive cross-modal validation mechanism occurs during the processing of multimedia materials (i.e., text combined with pictures).

Samples

Participants (Experiment 1: N = 146; Experiment 2: N = 235) were recruited via Prolific.

Methods

We used the epistemic Stroop paradigm (Richter et al., 2009), which makes use of the fact that the passive validation mechanism induces positive or negative response tendencies that can interfere with the processing of an unrelated task if it requires an opposite response. Participants received either matching (valid) or mismatching (invalid) text-picture stimuli. Following each stimulus, participants performed an unrelated probe-word task reacting to the probe words “wrong” or “right”. The dependent variables were reaction time and error rates in the unrelated probe-word task. Experiment 1 used one-sentence-picture stimuli, while Experiment 2 used longer text-segments-picture stimuli.

Results

Linear mixed-effects models showed interactions of validity and probe word for reaction times (Experiments 1 & 2) and error rates (Experiment 1). Post-hoc comparisons indicated prolonged reaction times or higher error rates when the probe word task required a response opposite to the outcome of the validation process.

Conclusions

This study is the first to demonstrate that a passive cross-modal validation mechanism checks the consistency between written text and accompanying pictures. We discuss the implications of these findings for understanding information processing in multimedia contexts.

在处理多媒体资料时,是否会出现被动的跨模态验证过程?
背景在文本理解研究中,已经观察到一种被动验证机制,它可以检查传入文本信息与先前文本信息(或先前知识)之间的一致性、方法我们使用了认识论 Stroop 范式(Richter 等人,2009 年),该范式利用了被动验证机制会诱发积极或消极反应倾向这一事实,如果不相关的任务需要相反的反应,这种反应倾向就会干扰任务的处理。参与者会收到匹配(有效)或不匹配(无效)的文字图片刺激。在每次刺激之后,受试者都要执行一项无关的探究词任务,对探究词 "错误 "或 "正确 "做出反应。因变量是无关探究词任务中的反应时间和错误率。结果线性混合效应模型显示,有效性和探究词对反应时间(实验 1 & 2)和错误率(实验 1)有交互作用。事后比较表明,当探测词任务要求的反应与验证过程的结果相反时,反应时间延长或错误率升高。结论这项研究首次证明了被动的跨模态验证机制可以检查书面文字与所附图片之间的一致性。我们讨论了这些发现对于理解多媒体环境下信息处理的意义。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
11.30
自引率
4.80%
发文量
109
期刊介绍: As an international, multi-disciplinary, peer-refereed journal, Learning and Instruction provides a platform for the publication of the most advanced scientific research in the areas of learning, development, instruction and teaching. The journal welcomes original empirical investigations. The papers may represent a variety of theoretical perspectives and different methodological approaches. They may refer to any age level, from infants to adults and to a diversity of learning and instructional settings, from laboratory experiments to field studies. The major criteria in the review and the selection process concern the significance of the contribution to the area of learning and instruction, and the rigor of the study.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信