Psychological methods最新文献

{"title":"Data aggregation can lead to biased inferences in Bayesian linear mixed models and Bayesian analysis of variance.","authors":"Daniel J Schad, Bruno Nicenboim, Shravan Vasishth","doi":"10.1037/met0000621","DOIUrl":"10.1037/met0000621","url":null,"abstract":"<p><p>Bayesian linear mixed-effects models (LMMs) and Bayesian analysis of variance (ANOVA) are increasingly being used in the cognitive sciences to perform null hypothesis tests, where a null hypothesis that an effect is zero is compared with an alternative hypothesis that the effect exists and is different from zero. While software tools for Bayes factor null hypothesis tests are easily accessible, how to specify the data and the model correctly is often not clear. In Bayesian approaches, many authors use data aggregation at the by-subject level and estimate Bayes factors on aggregated data. Here, we use simulation-based calibration for model inference applied to several example experimental designs to demonstrate that, as with frequentist analysis, such null hypothesis tests on aggregated data can be problematic in Bayesian analysis. Specifically, when random slope variances differ (i.e., violated sphericity assumption), Bayes factors are too conservative for contrasts where the variance is small and they are too liberal for contrasts where the variance is large. Running Bayesian ANOVA on aggregated data can-if the sphericity assumption is violated-likewise lead to biased Bayes factor results. Moreover, Bayes factors for by-subject aggregated data are biased (too liberal) when random item slope variance is present but ignored in the analysis. These problems can be circumvented or reduced by running Bayesian LMMs on nonaggregated data such as on individual trials, and by explicitly modeling the full random effects structure. Reproducible code is available from https://osf.io/mjf47/. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":"1133-1168"},"PeriodicalIF":7.8,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139564771","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A simple Monte Carlo method for estimating power in multilevel designs.","authors":"Craig K Enders, Brian T Keller, Michael P Woller","doi":"10.1037/met0000614","DOIUrl":"10.1037/met0000614","url":null,"abstract":"<p><p>Estimating power for multilevel models is complex because there are many moving parts, several sources of variation to consider, and unique sample sizes at Level 1 and Level 2. Monte Carlo computer simulation is a flexible tool that has received considerable attention in the literature. However, much of the work to date has focused on very simple models with one predictor at each level and one cross-level interaction effect, and approaches that do not share this limitation require users to specify a large set of population parameters. The goal of this tutorial is to describe a flexible Monte Carlo approach that accommodates a broad class of multilevel regression models with continuous outcomes. Our tutorial makes three important contributions. First, it allows any number of within-cluster effects, between-cluster effects, covariate effects at either level, cross-level interactions, and random coefficients. Moreover, we do not assume orthogonal effects, and predictors can correlate at either level. Second, our approach accommodates models with multiple interaction effects, and it does so with exact expressions for the variances and covariances of product random variables. Finally, our strategy for deriving hypothetical population parameters does not require pilot or comparable data. Instead, we use intuitive variance-explained effect size expressions to reverse-engineer solutions for the regression coefficients and variance components. We describe a new R package mlmpower that computes these solutions and automates the process of generating artificial data sets and summarizing the simulation results. The online supplemental materials provide detailed vignettes that annotate the R scripts and resulting output. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":"980-996"},"PeriodicalIF":7.8,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"92156263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Empirical selection of referent variables: Comparing multiple-indicator multiple-cause-interaction modeling and moderated nonlinear factor analysis.","authors":"Cheng-Hsien Li","doi":"10.1037/met0000613","DOIUrl":"10.1037/met0000613","url":null,"abstract":"<p><p>The fulfillment of measurement invariance/equivalence is considered a prerequisite for meaningfully proceeding with substantive cross-group comparisons. In the multiple-group confirmatory factor analysis approach, one model identification issue has unfortunately received little attention: the specification of a referent variable in the test of measurement invariance. A multiple-indicator multiple-cause (MIMIC) model with moderated effects (i.e., MIMIC-interaction modeling; Woods & Grimm, 2011) and a moderated nonlinear factor analysis (MNLFA; Bauer, 2017) model for detecting uniform and nonuniform measurement inequivalences in tandem were proposed to identify credible referent variables. The performance of two search strategies, constrained and free baseline models, and MIMIC-interaction and MNLFA methodologies were evaluated in a Monte Carlo simulation. Effects of different configurations of the number of inequivalent variables, type and magnitude of inequivalence, magnitude of group differences in factor means and variances, and sample size in combination with each search strategy were determined. Results showed that the constrained baseline model strategy generally outperformed the free baseline model strategy for identifying credible referent variables, functioning well when up to one-third of the observed variables were noninvariant. Moreover, MNLFA performed better than MIMIC-interaction modeling for the selection of referent variables across nearly all conditions investigated in the study. The superiority of MNLFA over MIMIC-interaction modeling was specifically evident in the models with relatively small samples, large between-group latent variance differences, or a combination of both. An empirical example was presented to demonstrate the applicability of MNLFA with the constrained baseline model strategy for referent variable selection. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":"1056-1078"},"PeriodicalIF":7.8,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"92156265","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"How to synthesize randomized controlled trial data with meta-analytic structural equation modeling: A comparison of various d-to-rpb conversions.","authors":"Hannelies de Jonge, Kees-Jan Kan, Frans J Oort, Suzanne Jak","doi":"10.1037/met0000790","DOIUrl":"https://doi.org/10.1037/met0000790","url":null,"abstract":"<p><p>Meta-analytic structural equation modeling (MASEM) allows a researcher to simultaneously examine multiple relations among variables by fitting a structural equation model to summary statistics from multiple studies. Consider, for example, a mediation model with a predictor (<i>X</i>), mediator (<i>M</i>), and outcome variable (<i>Y</i>). In such a model, <i>X</i> can be a dichotomous variable, allowing researchers to examine the direct and indirect effects of an intervention as in randomized controlled trials (RCTs). However, the natural choice of a meta-analysis of RCTs would involve standardized mean differences as effect sizes, whereas MASEM requires correlation matrices as input. This can be solved by converting standardized mean differences (Cohen's <i>d</i> or Hedges' <i>g</i>) to point-biserial correlations (<i>r</i>_pb). Possible conversion formulas vary across publications and conversion tools, and it is unclear which one is most appropriate for use in MASEM. The aim of this article is to describe and evaluate several conversions of standardized mean differences to point-biserial correlations in the context of RCTs. We investigate the impact of the usage of various conversions on MASEM parameter estimation using the R package metaSEM in a simulation study, varying the ratio of group sample sizes, number of primary studies, sample sizes, and missingness. The results show that a relatively unknown <i>d</i>-to-<i>r</i>_pb conversion generally performs best. However, this conversion formula is not implemented in the mainstream conversion tools. We developed a user-friendly web application entitled Effect Size Calculator and Converter (https://hdejonge.shinyapps.io/ESCACO) that converts the user's primary study statistics into an effect size suitable for use in MASEM. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.8,"publicationDate":"2025-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145192491","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Crowdsourcing multiverse analyses to explore the impact of different data-processing and analysis decisions: A tutorial.","authors":"Tom Heyman,Ekaterina Pronizius,Savannah C Lewis,Oguz A Acar,Matúš Adamkovič,Ettore Ambrosini,Jan Antfolk,Krystian Barzykowski,Ernest Baskin,Carlota Batres,Leanne Boucher,Jordane Boudesseul,Eduard Brandstätter,W Matthew Collins,Dušica Filipović Ðurđević,Ciara Egan,Vanessa Era,Paulo Ferreira,Chiara Fini,Patricia Garrido-Vásquez,Hendrik Godbersen,Pablo Gomez,Aurelien Graton,Necdet Gurkan,Zhiran He,Dave C Johnson,Pavol Kačmár,Chris Koch,Marta Kowal,Tomas Kratochvil,Marco Marelli,Fernando Marmolejo-Ramos,Martín Martínez,Alan Mattiassi,Nicholas P Maxwell,Maria Montefinese,Coby Morvinski,Maital Neta,Yngwie A Nielsen,Sebastian Ocklenburg,Jaš Onič,Marietta Papadatou-Pastou,Adam J Parker,Mariola Paruzel-Czachura,Yuri G Pavlov,Manuel Perea,Gerit Pfuhl,Tanja C Roembke,Jan P Röer,Timo B Roettger,Susana Ruiz-Fernandez,Kathleen Schmidt,Cynthia S Q Siew,Christian K Tamnes,Jack E Taylor,Rémi Thériault,José L Ulloa,Miguel A Vadillo,Michael E W Varnum,Martin R Vasilev,Steven Verheyen,Giada Viviani,Sebastian Wallot,Yuki Yamada,Yueyuan Zheng,Erin M Buchanan","doi":"10.1037/met0000770","DOIUrl":"https://doi.org/10.1037/met0000770","url":null,"abstract":"When processing and analyzing empirical data, researchers regularly face choices that may appear arbitrary (e.g., how to define and handle outliers). If one chooses to exclusively focus on a particular option and conduct a single analysis, its outcome might be of limited utility. That is, one remains agnostic regarding the generalizability of the results, because plausible alternative paths remain unexplored. A multiverse analysis offers a solution to this issue by exploring the various choices pertaining to data-processing and/or model building, and examining their impact on the conclusion of a study. However, even though multiverse analyses are arguably less susceptible to biases compared to the typical single-pathway approach, it is still possible to selectively add or omit pathways. To address this issue, we outline a novel, more principled approach to conducting multiverse analyses through crowdsourcing. The approach is detailed in a step-by-step tutorial to facilitate its implementation. We also provide a worked-out illustration featuring the Semantic Priming Across Many Languages project, thereby demonstrating its feasibility and its ability to increase objectivity and transparency. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":"1 1","pages":""},"PeriodicalIF":7.0,"publicationDate":"2025-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145078142","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploratory graph analysis trees-A network-based approach to investigate measurement invariance with numerous covariates.","authors":"David Goretzko,Philipp Sterner","doi":"10.1037/met0000796","DOIUrl":"https://doi.org/10.1037/met0000796","url":null,"abstract":"When comparing relationships between latent variables across groups, measurement invariance (MI) needs to be established to ensure that the test results are valid and meaningful conclusions can be drawn. Common tests of MI are not ideal for investigating many groups and are of limited value during the development of measurement models. In addition, popular network-based alternatives to latent variable modeling lack established methods for MI testing. Therefore, we propose exploratory graph analysis trees (EGA trees) that apply the idea of model-based recursive partitioning to correlation matrices and combine it with EGA-which can be used instead of exploratory factor analysis. In a simulation study, we test the approach regarding its ability to detect configural or metric noninvariance in common factor models given numerous covariates and illustrate its usefulness in conditions with severe violations of configural invaraince based on the diverging number of factors. The results demonstrate that EGA trees can be a valuable tool for the exploration of MI when constructing scales and working on measurement models. We provide R functions within the R package EFAtree to easily implement EGA trees. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":"67 1","pages":""},"PeriodicalIF":7.0,"publicationDate":"2025-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145058900","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"On the uncanny relationship between nonnormality and moderated multiple regression.","authors":"Oscar L Olvera Astivia,Xijuan Zhang,Edward Kroc,Bruno D Zumbo","doi":"10.1037/met0000797","DOIUrl":"https://doi.org/10.1037/met0000797","url":null,"abstract":"Moderated multiple regression is one of the most established, popular methods to model nonlinear associations in social sciences. A mostly unacknowledged fact is that a particular type of nonnormality can make the coefficient capturing this association nonzero. To further understand this connection, a theoretical investigation was conducted. A generalization of Isserlis' theorem from multivariate normal densities to all elliptical densities is presented. Through this generalization, it was found that the family of elliptical densities (which includes the multivariate normal) cannot generate a product-interaction term. Moreover, asymmetry in lower and/or higher dimensions can induce a product-interaction term. Special case studies are presented where the variables are unidimensional symmetric, but jointly nonsymmetric, resulting in a moderated multiple regression model. A call is made for researchers to think carefully and decide when they have a true interaction term, theorized a priori, and when nonnormality is mimicking an interaction effect. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":"14 1","pages":""},"PeriodicalIF":7.0,"publicationDate":"2025-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145018258","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Truncating the likelihood allows outlier exclusion without overestimating the evidence in the Bayes factor t test.","authors":"Henrik R. Godmann, František Bartoš, Eric-Jan Wagenmakers","doi":"10.1037/met0000782","DOIUrl":"https://doi.org/10.1037/met0000782","url":null,"abstract":"","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":"23 1","pages":""},"PeriodicalIF":7.0,"publicationDate":"2025-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144910621","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Coefficient of agreement between two raters corrected for category prevalence: Alternative to kappa.","authors":"Rashid Saif Almehrizi","doi":"10.1037/met0000732","DOIUrl":"https://doi.org/10.1037/met0000732","url":null,"abstract":"","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":"40 1","pages":""},"PeriodicalIF":7.0,"publicationDate":"2025-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144899765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Constructing a binary prediction model with incomplete data: Variable selection to balance fairness and precision.","authors":"He Ren, Chun Wang, Gongjun Xu, David J Weiss","doi":"10.1037/met0000786","DOIUrl":"10.1037/met0000786","url":null,"abstract":"<p><p>The statistical and pragmatic tension between explanation and prediction is well recognized in psychology. Yarkoni and Westfall (2017) suggested focusing more on predictions, which will ultimately produce better calibrated interpretations. Variable selection methods, such as regularization, are strongly recommended because it will help construct interpretable models while optimizing prediction accuracy. However, when the data contain a nonignorable proportion of missingness, variable selection and model building via penalized regression methods are not straightforward. What further complicates the analysis protocol is when the model performance is evaluated on both prediction accuracy and fairness, the latter is of increasing attention when the predictive outcome has societal implications. This study explored two methods for variable selection with incomplete data: the bootstrap imputation-stability selection (BI-SS) method and the stacked elastic net (SENET) method. Both methods work with multiply imputed data sets but in different ways. BI-SS implements variable selection separately on each imputed bootstrap data set and aggregates the results via stability selection, while SENET stacks all imputed data sets and fits a single pooled model. We thoroughly evaluated their performance using a suite of metrics (including area under the curve, F1 score, and fairness criteria) via three increasingly complex simulation studies. Results reveal that while BI-SS and SENET methods perform almost equally well in settings with generalized linear models, only BI-SS fares well with nested data design because of high computation demand in fitting the regularized generalized linear mixed effects models. Finally, we demonstrated both methods with an example using rich electronic health data. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.8,"publicationDate":"2025-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12356495/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144856196","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}