{"title":"Robust estimation of the latent trait in graded response models.","authors":"Audrey Filonczuk, Ying Cheng","doi":"10.3758/s13428-024-02574-2","DOIUrl":"10.3758/s13428-024-02574-2","url":null,"abstract":"<p><p>Aberrant responses (e.g., careless responses, miskeyed items, etc.) often contaminate psychological assessments and surveys. Previous robust estimators for dichotomous IRT models have produced more accurate latent trait estimates with data containing response disturbances. However, for widely used Likert-type items with three or more response categories, a robust estimator for estimating latent traits does not exist. We propose a robust estimator for the graded response model (GRM) that can be applied to Likert-type items. Two weighting mechanisms for downweighting \"suspicious\" responses are considered: the Huber and the bisquare weight functions. Simulations reveal the estimator reduces bias for various test lengths, numbers of response categories, and types of response disturbances. The reduction in bias and stable standard errors suggests that the robust estimator for the GRM is effective in counteracting the harmful effects of response disturbances and providing more accurate scores on psychological assessments. The robust estimator is then applied to data from the Big Five Inventory-2 (Ober et al., 2021) to demonstrate its use. Potential applications and implications are discussed.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 1","pages":"55"},"PeriodicalIF":4.6,"publicationDate":"2025-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142982536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A response time-based mixture item response theory model for dynamic item-response strategies.","authors":"Sijia Huang, Jinwen Luo, Minjeong Jeon","doi":"10.3758/s13428-024-02555-5","DOIUrl":"10.3758/s13428-024-02555-5","url":null,"abstract":"<p><p>Educational researchers have a long-lasting interest in the strategies examinees employ when responding to items in an assessment. Mixture item response theory (IRT) modeling is a popular class of approaches to studying examinees' item-response strategies. In the present study, we introduce a response time (RT)-based mixture IRT model for flexible modeling of examinee-and-item-specific item-response strategies. We posit that examinees may alternate between ability-based and non-ability-based strategies across different test items. Our proposed model identifies such within-examinee strategy switches without the need to predefine the non-ability-based strategies. Instead, our proposed approach allows for inferring the nature of these strategies from model parameter estimates. We illustrated the proposed approach using empirical data from PISA 2018 Science test and evaluated it through simulation studies. We concluded the article with discussions of limitations and future research directions.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 1","pages":"54"},"PeriodicalIF":4.6,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142943472","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Suzanne Jak, Terrence D Jorgensen, Mathilde G E Verdam, Frans J Oort, Louise Elffers
{"title":"Author Correction: Analytical power calculations for structural equation modeling: A tutorial and Shiny app.","authors":"Suzanne Jak, Terrence D Jorgensen, Mathilde G E Verdam, Frans J Oort, Louise Elffers","doi":"10.3758/s13428-024-02571-5","DOIUrl":"10.3758/s13428-024-02571-5","url":null,"abstract":"","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 1","pages":"53"},"PeriodicalIF":4.6,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11717876/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142943473","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Testing measurement invariance in a conditional likelihood framework by considering multiple covariates simultaneously.","authors":"Clemens Draxler, Andreas Kurz","doi":"10.3758/s13428-024-02551-9","DOIUrl":"10.3758/s13428-024-02551-9","url":null,"abstract":"<p><p>This article addresses the problem of measurement invariance in psychometrics. In particular, its focus is on the invariance assumption of item parameters in a class of models known as Rasch models. It suggests a mixed-effects or random intercept model for binary data together with a conditional likelihood approach of both estimating and testing the effects of multiple covariates simultaneously. The procedure can also be viewed as a multivariate multiple regression analysis which can be applied in longitudinal designs to investigate effects of covariates over time or different experimental conditions. This work also derives four statistical tests based on asymptotic theory and a parameter-free test suitable in small sample size scenarios. Finally, it outlines generalizations for categorical data in more than two categories. All procedures are illustrated on real-data examples from behavioral research and on a hypothetical data example related to clinical research in a longitudinal design.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 1","pages":"50"},"PeriodicalIF":4.6,"publicationDate":"2025-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11711259/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142943479","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Quantifying sighting dominance using on-display projections of monocular and binocular views.","authors":"Giuseppe Notaro, Uri Hasson","doi":"10.3758/s13428-024-02512-2","DOIUrl":"10.3758/s13428-024-02512-2","url":null,"abstract":"<p><p>Sighting dominance is an important behavioral property which has been difficult to measure quantitatively with high precision. We developed a measurement method that is grounded in a two-camera model that satisfies these aims. Using a simple alignment task, this method quantifies sighting ocular dominance during binocular viewing, identifying each eye's relative contribution to binocular vision. The method involves placing a physical target between the viewer and a display. The viewer indicates the perceived target's projection on the display with both eyes open and with only one eye open. The relative location of the binocular projection in relation to the two monocular projections is the index of dominance. The method produces a continuous variable with robust test-retest reliability (ICC = 0.96). The unit of measurement for the computed quantity is physiologically grounded: it is proportional to the distance between the monocular projections, which we show is predictable from interpupillary distance and phoria. Comparisons with the classic 'hole in card' sighting dominance test show good agreement, but also hint at potential bias for determining right-eye dominance in the latter. Interestingly, we find that some individuals systematically demonstrate nearly balanced vision, a phenomenon previously construed as mixed dominance or noisy responses. We also present ways to quantify and mitigate sources of random noise in this measurement. Overall, this new method allows for precise estimation of sighting dominance during binocular viewing. We expect it will allow a more effective understanding of the neural basis of dominance and improved effectiveness when using sighting dominance as a covariate in more complex analyses.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 1","pages":"52"},"PeriodicalIF":4.6,"publicationDate":"2025-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142943475","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Semantic alignment: A measure to quantify the degree of semantic equivalence for English-Chinese translation equivalents based on distributional semantics.","authors":"Yufeng Liu, Shifa Chen, Yi Yang","doi":"10.3758/s13428-024-02527-9","DOIUrl":"10.3758/s13428-024-02527-9","url":null,"abstract":"<p><p>The degree of semantic equivalence of translation pairs is typically measured by asking bilinguals to rate the semantic similarity of them or comparing the number and meaning of dictionary entries. Such measures are subjective, labor-intensive, and unable to capture the fine-grained variation in the degree of semantic equivalence. Thompson et al. (in Nature Human Behaviour, 4(10), 1029-1038, 2020) propose a computational method to quantify the extent to which translation equivalents are semantically aligned by measuring the contextual use across languages. Here, we refine this method to quantify semantic alignment of English-Chinese translation equivalents using word2vec based on the proposal that the degree of similarity between the contexts associated with a word and those of its multiple translations vary continuously. We validate our measure using semantic alignment from GloVe and fastText, and data from two behavioral datasets. The consistency of semantic alignment induced across different models confirms the robustness of our method. We demonstrate that semantic alignment not only reflects human semantic similarity judgment of translation equivalents but also captures bilinguals' usage frequency of translations. We also show that our method is more cognitively plausible than Thompson et al.'s method. Furthermore, the correlations between semantic alignment and key psycholinguistic factors mirror those between human-rated semantic similarity and these variables, indicating that computed semantic alignment reflects the degree of semantic overlap of translation equivalents in the bilingual mental lexicon. We further provide the largest English-Chinese translation equivalent dataset to date, encompassing 50,088 translation pairs for 15,734 English words, their dominant Chinese translation equivalents, and their semantic alignment Rc values.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 1","pages":"51"},"PeriodicalIF":4.6,"publicationDate":"2025-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142943478","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Noémie Auclair-Ouellet, Alexandra Lavoie, Pascale Bédard, Alexandra Barbeau-Morrison, Patrick Drouin, Pascale Tremblay
{"title":"Expansion of the SyllabO+ corpus and database: Words, lemmas, and morphology.","authors":"Noémie Auclair-Ouellet, Alexandra Lavoie, Pascale Bédard, Alexandra Barbeau-Morrison, Patrick Drouin, Pascale Tremblay","doi":"10.3758/s13428-024-02582-2","DOIUrl":"10.3758/s13428-024-02582-2","url":null,"abstract":"<p><p>Having a detailed description of the psycholinguistic properties of a language is essential for conducting well-controlled language experiments. However, there is a paucity of databases for some languages and regional varieties, including Québec French. The SyllabO+ corpus was created to provide a complete phonological and syllabic analysis of a corpus of spoken Québec French. In the present study, the corpus was expanded with 41 additional speakers, bringing the total to 225. The analysis was also expanded to include three new databases: unique words, lemmas, and morphemes (inflectional, derivational, and compounds). Next, the internal structure of unique words was analyzed to identify roots, inflectional markers, and affixes, as well as the components of compounds. Additionally, a group of 441 speakers of Québec French provided semantic transparency ratings for 3764 derived words. Results from the semantic transparency judgment study show broad inter-individual variability for words of medium transparency. No influence of sociodemographic variables was found. Transparency ratings are coherent with studies showing the greater transparency of suffixed words compared to prefixed words. Results for participants who speak French as a second language support the association between second-language proficiency and morphological processing.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 1","pages":"47"},"PeriodicalIF":4.6,"publicationDate":"2025-01-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142943474","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Beyond Reality Image Collection (BRIC).","authors":"Noga Segal-Gordon, Yoav Bar-Anan","doi":"10.3758/s13428-024-02586-y","DOIUrl":"10.3758/s13428-024-02586-y","url":null,"abstract":"<p><p>The Beyond Reality Image Collection (BRIC) is a set of 648 photos, some painted by an artist and some generated by artificial intelligence. Unlike previous photosets, the BRIC focused on nonrealistic visuals. This collection includes abstract and non-abstract paintings and nonrealistic photographs depicting objects, scenes, animals, humans, and fantastical creatures with varying degrees of unreal elements. We collected evaluative ratings of the photos, using a convenience sample of 16,208 participants in a total of 25,321 sessions. We used multiple evaluation measures: binary positive/negative and like/dislike categorization, seven-point ratings on these attributes, both under no time pressure and under time pressure, and evaluative priming scores. The mean evaluation of the photos on the different measures was highly correlated, but some photos consistently elicited a discrepant evaluative reaction between the measures. The BRIC is a valuable resource for eliciting evaluative reactions and can contribute to research on evaluative processes and affective responses.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 1","pages":"49"},"PeriodicalIF":4.6,"publicationDate":"2025-01-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11706899/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142943480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sascha B Duken, Jun Moriya, Colette Hirsch, Marcella L Woud, Bram van Bockstaele, Elske Salemink
{"title":"Reliability and validity of four cognitive interpretation bias measures in the context of social anxiety.","authors":"Sascha B Duken, Jun Moriya, Colette Hirsch, Marcella L Woud, Bram van Bockstaele, Elske Salemink","doi":"10.3758/s13428-024-02576-0","DOIUrl":"10.3758/s13428-024-02576-0","url":null,"abstract":"<p><p>People with social anxiety disorder tend to interpret ambiguous social information in a negative rather than positive manner. Such interpretation biases may cause and maintain anxiety symptoms. However, there is considerable variability in the observed effects across studies, with some not finding a relationship between interpretation biases and social anxiety. Poor psychometric properties of interpretation bias measures may explain such inconsistent findings. We evaluated the internal consistency, test-retest reliability, convergent validity, and concurrent validity of four interpretation bias measures, ranging from more implicit and automatic to more explicit and reflective: the probe scenario task, the recognition task, the scrambled sentences task, and the interpretation and judgmental bias questionnaire. Young adults (N = 94) completed interpretation bias measures in two sessions separated by one week. Psychometric properties were poor for the probe scenario and not acceptable for the recognition task. The reliability of the scrambled sentences task and the interpretation and judgmental bias questionnaire was good, and they correlated highly with social anxiety and each other, supporting their concurrent and convergent validity. However, there are methodological challenges that should be considered when measuring interpretation biases, even if psychometric indices suggest high measurement validity. We also discuss likely reasons for poor psychometric properties of some tasks and suggest potential solutions to improve the assessment of implicit and automatic biases in social anxiety in future research.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 1","pages":"48"},"PeriodicalIF":4.6,"publicationDate":"2025-01-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11706852/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142943477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ana Levordashka, Mike Richardson, Rebecca J Hirst, Iain D Gilchrist, Danaë Stanton Fraser
{"title":"Trace: A research media player measuring real-time audience engagement.","authors":"Ana Levordashka, Mike Richardson, Rebecca J Hirst, Iain D Gilchrist, Danaë Stanton Fraser","doi":"10.3758/s13428-024-02522-0","DOIUrl":"10.3758/s13428-024-02522-0","url":null,"abstract":"<p><p>Measuring attention and engagement is essential for understanding a wide range of psychological phenomena. Advances in technology have made it possible to measure real-time attention to naturalistic stimuli, providing ecologically valid insight into temporal dynamics. We developed a research protocol called Trace, which records anonymous facial landmarks, expressions, and patterns of movement associated with engagement in screen-based media. Trace runs in a standard internet browser and resembles a contemporary media player. It is embedded in the open-source package PsychoJS (the JavaScript sister library of PsychoPy) hosted via Pavlovia, and can be integrated with a wide range of behavioral research methods. Developed over multiple iterations and tested with over 200 participants in three studies, including the official broadcast of a major theatre production, Trace is a powerful, user-friendly protocol allowing behavioral researchers to capture audience attention and engagement in screen-based media as part of authentic, ecologically valid audience experiences.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 1","pages":"44"},"PeriodicalIF":4.6,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11703984/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142943482","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}