Unveiling passive cross-modal reactivation and validation processes in the processing of multimedia material

IF 4.7 1区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Learning and Instruction Pub Date : 2025-03-20 DOI:10.1016/j.learninstruc.2025.102117

Anne Schüler , Pauline Frick

{"title":"Unveiling passive cross-modal reactivation and validation processes in the processing of multimedia material","authors":"Anne Schüler , Pauline Frick","doi":"10.1016/j.learninstruc.2025.102117","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>During reading, a passive reactivation process reactivates information from the discourse context, while a passive validation checks the consistency between reactivated and actual information.</div></div><div><h3>Aims</h3><div>Two pre-registered online studies examined whether passive cross-modal reactivation and validation processes occur during the processing of multimedia materials (text combined with pictures).</div></div><div><h3>Samples</h3><div>Participants were recruited via Clickworker (Experiment 1: <em>N</em> = 251; Experiment 2: <em>N</em> = 266).</div></div><div><h3>Methods</h3><div>We applied the epistemic Stroop paradigm (Richter et al., 2009), which utilizes automatic validation mechanisms that trigger positive or negative response tendencies. These tendencies can interfere with an unrelated task, requiring an opposite response. Participants were shown matching (valid) or mismatching (invalid) text-picture stimuli, with the text and picture components presented sequentially, interrupted by a calculation task, making reactivation necessary. Following each stimulus, participants performed an unrelated probe-word task responding to the words \"right\" or \"wrong\". Dependent variables were reaction time and error rates in the probe-word task. Experiment 1 used one-sentence-picture stimuli, while Experiment 2 used longer text segments with pictures.</div></div><div><h3>Results</h3><div>In both experiments, linear mixed-effects models revealed interactions between validity (valid vs. invalid stimulus) and probe word (right vs. wrong) for reaction times and error rates. Post-hoc analyses showed longer reaction times or higher error rates when participants encountered invalid stimuli before responding to the probe word “right”.</div></div><div><h3>Conclusions</h3><div>In this study, passive cross-modal reactivation and validation processes could be demonstrated together for the first time in multimedia processing. Implications for understanding information processing in multimedia contexts are discussed.</div></div>","PeriodicalId":48357,"journal":{"name":"Learning and Instruction","volume":"97 ","pages":"Article 102117"},"PeriodicalIF":4.7000,"publicationDate":"2025-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Learning and Instruction","FirstCategoryId":"95","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0959475225000416","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

During reading, a passive reactivation process reactivates information from the discourse context, while a passive validation checks the consistency between reactivated and actual information.

Aims

Two pre-registered online studies examined whether passive cross-modal reactivation and validation processes occur during the processing of multimedia materials (text combined with pictures).

Samples

Participants were recruited via Clickworker (Experiment 1: N = 251; Experiment 2: N = 266).

Methods

We applied the epistemic Stroop paradigm (Richter et al., 2009), which utilizes automatic validation mechanisms that trigger positive or negative response tendencies. These tendencies can interfere with an unrelated task, requiring an opposite response. Participants were shown matching (valid) or mismatching (invalid) text-picture stimuli, with the text and picture components presented sequentially, interrupted by a calculation task, making reactivation necessary. Following each stimulus, participants performed an unrelated probe-word task responding to the words "right" or "wrong". Dependent variables were reaction time and error rates in the probe-word task. Experiment 1 used one-sentence-picture stimuli, while Experiment 2 used longer text segments with pictures.

Results

In both experiments, linear mixed-effects models revealed interactions between validity (valid vs. invalid stimulus) and probe word (right vs. wrong) for reaction times and error rates. Post-hoc analyses showed longer reaction times or higher error rates when participants encountered invalid stimuli before responding to the probe word “right”.

Conclusions

In this study, passive cross-modal reactivation and validation processes could be demonstrated together for the first time in multimedia processing. Implications for understanding information processing in multimedia contexts are discussed.

查看原文本刊更多论文

揭示多媒体材料加工中的被动跨模态再激活和验证过程

在阅读过程中，被动再激活过程从话语语境中重新激活信息，而被动验证则检查重新激活的信息与实际信息之间的一致性。aimst2个预先注册的在线研究检查了被动跨模态再激活和验证过程是否发生在多媒体材料（文本与图片相结合）的处理过程中。参与者通过Clickworker(实验1:N = 251；实验2:N = 266)。方法采用认知Stroop范式（Richter et al., 2009），该范式利用触发积极或消极反应倾向的自动验证机制。这些倾向会干扰一个不相关的任务，需要相反的反应。参与者被显示匹配（有效）或不匹配（无效）的文本-图片刺激，文本和图片组件依次呈现，被计算任务打断，这使得重新激活是必要的。在每个刺激之后，参与者执行一个不相关的探测词任务，对“对”或“错”两个词做出反应。因变量为探测词任务的反应时间和错误率。实验1使用单句图片刺激，实验2使用较长的文字片段和图片刺激。结果在两个实验中，线性混合效应模型揭示了反应时间和错误率在效度（有效刺激与无效刺激）和探针词（对与错）之间的相互作用。事后分析表明，当参与者在对探测词“正确”做出反应之前遇到无效刺激时，反应时间更长，错误率更高。结论本研究首次在多媒体加工中证实了被动的跨模态再激活和验证过程。讨论了在多媒体环境中理解信息处理的含义。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Learning and Instruction Multiple-

CiteScore

11.30

自引率

4.80%

发文量

109

期刊介绍： As an international, multi-disciplinary, peer-refereed journal, Learning and Instruction provides a platform for the publication of the most advanced scientific research in the areas of learning, development, instruction and teaching. The journal welcomes original empirical investigations. The papers may represent a variety of theoretical perspectives and different methodological approaches. They may refer to any age level, from infants to adults and to a diversity of learning and instructional settings, from laboratory experiments to field studies. The major criteria in the review and the selection process concern the significance of the contribution to the area of learning and instruction, and the rigor of the study.