{"title":"Do passive cross-modal validation processes occur when processing multimedia materials?","authors":"Anne Schüler, Pauline Frick","doi":"10.1016/j.learninstruc.2024.101956","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><p>In text comprehension research, a passive validation mechanism has been observed that checks the consistency between incoming and previous text information (or prior knowledge).</p></div><div><h3>Aims</h3><p>In two pre-registered online studies, we investigated whether a passive cross-modal validation mechanism occurs during the processing of multimedia materials (i.e., text combined with pictures).</p></div><div><h3>Samples</h3><p>Participants (Experiment 1: <em>N</em> = 146; Experiment 2: <em>N</em> = 235) were recruited via Prolific.</p></div><div><h3>Methods</h3><p>We used the epistemic Stroop paradigm (Richter et al., 2009), which makes use of the fact that the passive validation mechanism induces positive or negative response tendencies that can interfere with the processing of an unrelated task if it requires an opposite response. Participants received either matching (valid) or mismatching (invalid) text-picture stimuli. Following each stimulus, participants performed an unrelated probe-word task reacting to the probe words “wrong” or “right”. The dependent variables were reaction time and error rates in the unrelated probe-word task. Experiment 1 used one-sentence-picture stimuli, while Experiment 2 used longer text-segments-picture stimuli.</p></div><div><h3>Results</h3><p>Linear mixed-effects models showed interactions of validity and probe word for reaction times (Experiments 1 & 2) and error rates (Experiment 1). Post-hoc comparisons indicated prolonged reaction times or higher error rates when the probe word task required a response opposite to the outcome of the validation process.</p></div><div><h3>Conclusions</h3><p>This study is the first to demonstrate that a passive cross-modal validation mechanism checks the consistency between written text and accompanying pictures. We discuss the implications of these findings for understanding information processing in multimedia contexts.</p></div>","PeriodicalId":48357,"journal":{"name":"Learning and Instruction","volume":null,"pages":null},"PeriodicalIF":4.7000,"publicationDate":"2024-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0959475224000835/pdfft?md5=80c4216addce7fcfd40de1fd5fb491d1&pid=1-s2.0-S0959475224000835-main.pdf","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Learning and Instruction","FirstCategoryId":"95","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0959475224000835","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0
Abstract
Background
In text comprehension research, a passive validation mechanism has been observed that checks the consistency between incoming and previous text information (or prior knowledge).
Aims
In two pre-registered online studies, we investigated whether a passive cross-modal validation mechanism occurs during the processing of multimedia materials (i.e., text combined with pictures).
Samples
Participants (Experiment 1: N = 146; Experiment 2: N = 235) were recruited via Prolific.
Methods
We used the epistemic Stroop paradigm (Richter et al., 2009), which makes use of the fact that the passive validation mechanism induces positive or negative response tendencies that can interfere with the processing of an unrelated task if it requires an opposite response. Participants received either matching (valid) or mismatching (invalid) text-picture stimuli. Following each stimulus, participants performed an unrelated probe-word task reacting to the probe words “wrong” or “right”. The dependent variables were reaction time and error rates in the unrelated probe-word task. Experiment 1 used one-sentence-picture stimuli, while Experiment 2 used longer text-segments-picture stimuli.
Results
Linear mixed-effects models showed interactions of validity and probe word for reaction times (Experiments 1 & 2) and error rates (Experiment 1). Post-hoc comparisons indicated prolonged reaction times or higher error rates when the probe word task required a response opposite to the outcome of the validation process.
Conclusions
This study is the first to demonstrate that a passive cross-modal validation mechanism checks the consistency between written text and accompanying pictures. We discuss the implications of these findings for understanding information processing in multimedia contexts.
期刊介绍:
As an international, multi-disciplinary, peer-refereed journal, Learning and Instruction provides a platform for the publication of the most advanced scientific research in the areas of learning, development, instruction and teaching. The journal welcomes original empirical investigations. The papers may represent a variety of theoretical perspectives and different methodological approaches. They may refer to any age level, from infants to adults and to a diversity of learning and instructional settings, from laboratory experiments to field studies. The major criteria in the review and the selection process concern the significance of the contribution to the area of learning and instruction, and the rigor of the study.