{"title":"How reliable are standard reading time analyses? Hierarchical bootstrap reveals substantial power over-optimism and scale-dependent Type I error inflation","authors":"Zachary J. Burchill , T. Florian Jaeger","doi":"10.1016/j.jml.2023.104494","DOIUrl":"10.1016/j.jml.2023.104494","url":null,"abstract":"<div><p>We investigate the statistical power and Type I error rate of the two most common approaches to reading time (RT) analyses: assuming normality of residuals and homogeneity of variance in raw or log-transformed RTs. We first show that the assumptions of such analyses—such as <em>t</em><span>-tests, ANOVAs, and linear mixed-effects models—are neither consistently met by raw RTs, nor by log-transformed RTs (or any other common power transforms, incl. inverse-transformed RTs). Only a non-power transform (log-shift) provides a decent fit for all data sets and data preparation steps we consider. We then compare the statistical power and Type I error rate for linear mixed-effects models over raw or log-transformed RTs. Previous studies on this matter relied on parametrically generated data. We show why this is problematic, and introduce as an alternative a hierarchical bootstrap approach over naturally distributed reading times. This approach yields substantially different—and arguably more informative—results than the parametric simulation approaches we compare it to. Our results suggests that it is time to heed the advice others have provided for reading research: for any but the simplest designs, we find both the rate of spurious significances and the rate of undetected true effects can </span><em>strongly</em> depend on the scale (e.g., raw or log-RTs) in which effects are assumed to be linear. Researchers should thus clearly motivate the choice of analysis based on theoretical grounds, assess the robustness of findings under different analysis approaches, and discuss potential mismatches between analyses. The R scripts and libraries shared in the accompanying OSF repo allow researchers to assess the reliability of their analyses via hierarchical bootstrap over their own data.</p></div>","PeriodicalId":16493,"journal":{"name":"Journal of memory and language","volume":"136 ","pages":"Article 104494"},"PeriodicalIF":4.3,"publicationDate":"2024-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139587570","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ruoyu Lu, Zeyu Li, Chenyu Yan, Tengfei Wang, Zhi Li
{"title":"Storage interference in working memory cannot be removed by attention","authors":"Ruoyu Lu, Zeyu Li, Chenyu Yan, Tengfei Wang, Zhi Li","doi":"10.1016/j.jml.2024.104498","DOIUrl":"https://doi.org/10.1016/j.jml.2024.104498","url":null,"abstract":"<div><p>In the present study, we examined the hypothesis that the storage interference in working memory can be removed by attention. A dual-task paradigm was employed in Experiment 1 and 2, in which participants performed a color memory task and an RSVP letter detection task concurrently. The cognitive load of the RSVP letter detection task and the storage interference caused by the RSVP letter detection task was manipulated independently. That is, the produced storage-interference difference between the low and high interference conditions was comparable between the low and high cognitive load conditions, whereas the available attentional resources were different under the two cognitive load conditions. Since there were more attentional resources in the low load condition, the removal hypothesis predicts that differences in recall performance between the low and high interference conditions should be larger in high load than in low load, i.e., there would be an interaction between load and interference. However, the results of the two experiments did not show such an interaction. In Experiment 3, we manipulated the time available for the removal mechanism to work while inducing both the storage interference and processing interference. The results showed no sign of interference removal. Thus, the present results provided solid evidence to challenge the removal hypothesis.</p></div>","PeriodicalId":16493,"journal":{"name":"Journal of memory and language","volume":"136 ","pages":"Article 104498"},"PeriodicalIF":4.3,"publicationDate":"2024-01-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139433697","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"What latent variable underlies confidence in lineup rejections?","authors":"Anne S. Yilmaz, John T. Wixted","doi":"10.1016/j.jml.2023.104493","DOIUrl":"10.1016/j.jml.2023.104493","url":null,"abstract":"<div><p>When a face is positively identified from a multi-person photo lineup, it is presumably the face that generates the strongest memory signal. In addition, confidence in a positive identification is presumably determined by the strength of the memory signal associated with that face. However, when no face generates a strong enough memory signal to be identified, the entire set of faces in the lineup is collectively rejected. What latent variable underlies confidence in a lineup rejection? One possibility is that the face that generates the strongest memory signal still determines confidence (i.e., the weaker that memory signal is, the more confidently the lineup is rejected). Another possibility is that confidence in a lineup rejection is determined by the average strength of the memory signals generated by the faces in the lineup (i.e., the weaker that average memory signal is, the more confidently the lineup is rejected). The reliance on an average signal has been proposed as a possible explanation for why the confidence-accuracy for lineup rejections tends to be weak. Here, we modified two existing signal-detection-based lineup models (the Independent Observations model and the Ensemble model) and fit them to multiple lineup datasets to investigate which decision variable underlies confidence in lineup rejections. Both models agree that confidence in a lineup rejection is based on the strongest memory signal in the lineup, not on the average signal. These model fits also revealed for the first time that the memory signals in a lineup are correlated, as they theoretically should be.</p></div>","PeriodicalId":16493,"journal":{"name":"Journal of memory and language","volume":"135 ","pages":"Article 104493"},"PeriodicalIF":4.3,"publicationDate":"2023-12-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139067688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The representation of agreement features in memory is updated during sentence processing: Evidence from verb-reflexive interactions","authors":"Maayan Keshev , Aya Meltzer-Asscher.","doi":"10.1016/j.jml.2023.104495","DOIUrl":"10.1016/j.jml.2023.104495","url":null,"abstract":"<div><p>The formation of linguistic dependencies is subject to memory interference. In this study, we ask whether memory representations are fixed, or whether they can be distorted and updated after their initial encoding. Models of Cue-Based Retrieval assume that memory representations are fixed. However, representational interference and rational inference models assume that memory contents can be edited. To examine this, we test how reflexive attraction is affected by preceding verbal agreement in Hebrew. Cue-Based Retrieval suggests that agreement on the verb can exaggerate the reflexive’s sensitivity to the distractor. In contrast, we propose that if memory representations can be edited, verbal agreement can alter the representation of the subject. This process would reduce vulnerability to distortions originating from the distractor. In two self-paced reading experiments and one forced-choice completion we find (i) decreased reflexive attraction when (grammatical or ungrammatical) agreement cues were available on the preceding verb; and (ii) a preference for reflexive forms matching the verb over forms matching the subject when the sentence included ungrammatical verbal agreement. These results suggest that comprehenders use featural information from the verb to recover properties of the subject. The findings are therefore consistent with a memory model where representations can be distorted and updated, as well as with rational inference about memory disruption.</p></div>","PeriodicalId":16493,"journal":{"name":"Journal of memory and language","volume":"135 ","pages":"Article 104495"},"PeriodicalIF":4.3,"publicationDate":"2023-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139025369","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Sources and goals in memory and language: Fragility and robustness in event representation","authors":"Yiran Chen , John Trueswell , Anna Papafragou","doi":"10.1016/j.jml.2023.104475","DOIUrl":"10.1016/j.jml.2023.104475","url":null,"abstract":"<div><p>Previous research has demonstrated an asymmetry between Sources and Goals in people’s linguistic and non-linguistic encoding of motion events: when describing events such as a fairy going from a tree to a flower, people mention the Goal (“to a flower”) more often than the Source (“from a tree”); similarly, people are better at detecting Goal than Source changes in memory tests. However, all prior work used a single task to probe memory of Sources and Goals and thus left the nature of the fragility of event components open. Here, we probed memory for Sources and Goals using either a Same-different or a Forced-choice task after participants passively viewed (Experiment 1), viewed and described (Experiment 2) or viewed and heard descriptions of (Experiment 3) the same set of motion events. We robustly replicated the linguistic Source-Goal asymmetry. However, across encoding contexts, the memory asymmetry persisted in the Same-different task but <em>disappeared</em><span> in the Forced-choice task. The Same-different task results did not change even when participants were explicitly asked to attend to Sources (Experiment 4a) and when motion trajectory was removed at test (Experiment 4b), ruling out a purely test-expectation account for the cross-task effect. We conclude that Sources of motion, even when not mentioned in language nor successfully retrieved at memory test, are nevertheless represented as part of a motion event, and their detailed representation can be reinstated at aided retrieval contexts. Our data clarify the nature of event representation and suggest a fine-grained homology between language and event memory.</span></p></div>","PeriodicalId":16493,"journal":{"name":"Journal of memory and language","volume":"135 ","pages":"Article 104475"},"PeriodicalIF":4.3,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139025365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Victor Kuperman , Sascha Schroeder , Daniil Gnetov
{"title":"Word length and frequency effects on text reading are highly similar in 12 alphabetic languages","authors":"Victor Kuperman , Sascha Schroeder , Daniil Gnetov","doi":"10.1016/j.jml.2023.104497","DOIUrl":"10.1016/j.jml.2023.104497","url":null,"abstract":"<div><p>Reading research robustly finds that shorter and more frequent words are recognized faster and skipped more often than longer and less frequent words. An empirical question that has not been tested yet is whether languages within the same writing system would produce similarly strong length and frequency effects or whether typological differences between written languages would cause those effects to vary systematically in their magnitude. We analyzed text reading eye-movement data in 12 alphabetic languages from the Multilingual Eye-Movement Corpus (MECO). The languages varied substantially in their word length and frequency distributions as a function of their orthographic depth and morpho-syntactic type. Yet, the effects of word length and frequency on fixation durations and skipping rate were highly similar in size between the languages. This finding suggests a high degree of cross-linguistic universality in the readers’ behavioral response to linguistic complexity (indexed by word length) and the amount of experience with the word (indexed by word frequency). These findings run counter to influential theories of single word recognition, which predict orthographic depth of a language to modulate the size of these benchmark effects. They also facilitate development of cross-linguistically generalizable computational models of eye-movement control in reading.</p></div>","PeriodicalId":16493,"journal":{"name":"Journal of memory and language","volume":"135 ","pages":"Article 104497"},"PeriodicalIF":4.3,"publicationDate":"2023-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138819758","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Maximilian M. Rabe , Dario Paape , Daniela Mertzen , Shravan Vasishth , Ralf Engbert
{"title":"SEAM: An integrated activation-coupled model of sentence processing and eye movements in reading","authors":"Maximilian M. Rabe , Dario Paape , Daniela Mertzen , Shravan Vasishth , Ralf Engbert","doi":"10.1016/j.jml.2023.104496","DOIUrl":"10.1016/j.jml.2023.104496","url":null,"abstract":"<div><p>Models of eye-movement control during reading, developed largely within psychology, usually focus on visual, attentional, lexical, and motor processes but neglect post-lexical language processing; by contrast, models of sentence comprehension processes, developed largely within psycholinguistics, generally focus only on post-lexical language processes. We present a model that combines these two research threads, by integrating eye-movement control and sentence processing. Developing such an integrated model is extremely challenging and computationally demanding, but such an integration is an important step toward complete mathematical models of natural language comprehension in reading. We combine the SWIFT model of eye-movement control (Seelig et al., 2023) with key components of the Lewis and Vasishth sentence processing model (Lewis and Vasishth, 2005). This integration becomes possible, for the first time, due in part to recent advances in successful parameter identification in dynamical models, which allows us to investigate profile log-likelihoods for individual model parameters. We present a fully implemented proof-of-concept model demonstrating how such an integrated model can be achieved; our approach includes Bayesian model inference with Markov Chain Monte Carlo (MCMC) sampling as a key computational tool. The integrated Sentence-Processing and Eye-Movement Activation-Coupled Model (SEAM) can successfully reproduce eye movement patterns that arise due to similarity-based interference in reading. To our knowledge, this is the first-ever integration of a complete process model of eye-movement control with linguistic dependency completion processes in sentence comprehension. In future work, this proof of concept model will need to be evaluated using a comprehensive set of benchmark data.</p></div>","PeriodicalId":16493,"journal":{"name":"Journal of memory and language","volume":"135 ","pages":"Article 104496"},"PeriodicalIF":4.3,"publicationDate":"2023-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0749596X23000955/pdfft?md5=e957b46c6324edb315808fc2e1ef2d43&pid=1-s2.0-S0749596X23000955-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138820006","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Do readers here what they sea?: Effects of lexicality, predictability, and individual differences on the phonological preview benefit","authors":"Sara Milligan, Elizabeth R. Schotter","doi":"10.1016/j.jml.2023.104480","DOIUrl":"https://doi.org/10.1016/j.jml.2023.104480","url":null,"abstract":"<div><p>For decades, researchers have debated whether readers benefit from translating visual word forms into phonological codes. A focus of this debate has been on the earliest moments of processing when a word is perceived in parafoveal vision (i.e., phonological preview benefit). A recent meta-analysis (<span>Vasilev et al., 2019</span>) concluded that the phonological preview benefit may be small and unreliable but they did not take into account potentially important stimulus-level or participant-level factors that varied across the included studies. Therefore, we conducted two well-powered experiments that systematically investigated the effects of sentence constraint, preview lexicality, and participant language skills on the phonological preview benefit effect. We found phonological preview benefits that were larger in high versus low constraint sentences, larger for words than pseudowords, and larger for better spellers. We conclude that phonological codes do facilitate early word recognition during reading, but that the phonological preview benefit magnitude depends on subject- and stimulus-level factors.</p></div>","PeriodicalId":16493,"journal":{"name":"Journal of memory and language","volume":"135 ","pages":"Article 104480"},"PeriodicalIF":4.3,"publicationDate":"2023-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138549960","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mariana Vega-Mendoza , Iva Ivanova , Janet F. McLean , Martin J. Pickering , Holly P. Branigan
{"title":"Lexically-specific syntactic restrictions in second-language speakers","authors":"Mariana Vega-Mendoza , Iva Ivanova , Janet F. McLean , Martin J. Pickering , Holly P. Branigan","doi":"10.1016/j.jml.2023.104470","DOIUrl":"https://doi.org/10.1016/j.jml.2023.104470","url":null,"abstract":"<div><p>In two structural priming experiments, we investigated the representations of lexically-specific syntactic restrictions of English verbs for highly proficient and immersed second language (L2) speakers of English. We considered the interplay of two possible mechanisms: generalization from the first language (L1) and statistical learning within the L2 (both of abstract structure and of lexically-specific information). In both experiments, L2 speakers with either Germanic or Romance languages as L1 were primed to produce dispreferred double-object structures involving non-alternating dative verbs. Priming occurred from ungrammatical double-object primes involving different non-alternating verbs (Experiment 1) and from grammatical primes involving alternating verbs (Experiment 2), supporting abstract statistical learning within the L2. However, we found no differences between L1-Germanic speakers (who have the double-object structure in their L1) and L1-Romance speakers (who do not), inconsistent with the prediction for between-group differences of the L1-generalization account. Additionally, L2 speakers in Experiment 2 showed a lexical boost: There was stronger priming after (dispreferred) non-alternating same-verb double-object primes than after (grammatical) alternating different-verb primes. Such lexically-driven persistence was also shown by L1 English speakers (Ivanova, Pickering, McLean, Costa, & Branigan, 2012) and may underlie statistical learning of lexically-dependent structural regularities. We conclude that lexically-specific syntactic restrictions in highly proficient and immersed L2 speakers are shaped by statistical learning (both abstract and lexically-specific) within the L2, but not by generalization from the L1.</p></div>","PeriodicalId":16493,"journal":{"name":"Journal of memory and language","volume":"134 ","pages":"Article 104470"},"PeriodicalIF":4.3,"publicationDate":"2023-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0749596X23000694/pdfft?md5=68cbe1c90d4cd5aa4b5afbc2cb46cbfb&pid=1-s2.0-S0749596X23000694-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138448500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Language comprehenders are sensitive to multiple states of semantically similar objects","authors":"Oleksandr V. Horchak, Margarida V. Garrido","doi":"10.1016/j.jml.2023.104478","DOIUrl":"https://doi.org/10.1016/j.jml.2023.104478","url":null,"abstract":"<div><p>The present research shows that language comprehenders are sensitive to multiple states of target and semantically related objects. In Experiments 1 to 2B, participants (total <em>N</em> = 273) read sentences that either implied a minimal change of an object’s state (e.g., “Jane <em>chose</em> a mango”) or a substantial change (e.g., “Jane <em>stepped</em> on a mango”) and then verified whether a subsequently pictured object was mentioned in the sentence. Crucially, the picture either showed the original/modified state of an object that was mentioned in the sentence (e.g., “mango” in Experiment 1) or not (e.g., “banana” in Experiments 2A and 2B). The results of Experiment 1 demonstrated that the objects in a modified state were verified faster when a sentence implied a substantial state-change rather than a minimal state-change. In contrast, the reverse was true for the objects in the original state. Importantly, verification latencies of pictures depicting original and modified states of an object in the substantial state-change condition were approximately similar, thus suggesting that language comprehenders maintain multiple representations of an object in different states. The results of Experiments 2A and 2B revealed that when participants had to indicate that a pictured object (e.g., banana) was not mentioned in the sentence, their verification latencies were slowed down when the sentence contained a semantically related item (e.g., mango) and described this item as being changed substantially by the action. However, these verification latencies varied continuously with the degree of change: the more dissimilar the states of a semantically related item, the less time participants needed to verify a pictured object. The results are discussed through the prism of theories emphasizing dynamic views of event cognition.</p></div>","PeriodicalId":16493,"journal":{"name":"Journal of memory and language","volume":"135 ","pages":"Article 104478"},"PeriodicalIF":4.3,"publicationDate":"2023-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0749596X23000773/pdfft?md5=f159957dfd86618c4d6a6f0b6fa7bd43&pid=1-s2.0-S0749596X23000773-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138412312","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}