Psychological methodsPub Date : 2025-06-01Epub Date: 2023-06-08DOI: 10.1037/met0000552
Holger Brandt, Siyuan Marco Chen, Daniel J Bauer
{"title":"Bayesian penalty methods for evaluating measurement invariance in moderated nonlinear factor analysis.","authors":"Holger Brandt, Siyuan Marco Chen, Daniel J Bauer","doi":"10.1037/met0000552","DOIUrl":"10.1037/met0000552","url":null,"abstract":"<p><p>Measurement invariance (MI) is one of the main psychometric requirements for analyses that focus on potentially heterogeneous populations. MI allows researchers to compare latent factor scores across persons from different subgroups, whereas if a measure is not invariant across all items and persons then such comparisons may be misleading. If full MI does not hold further testing may identify problematic items showing differential item functioning (DIF). Most methods developed to test DIF focused on simple scenarios often with comparisons across two groups. In practical applications, this is an oversimplification if many grouping variables (e.g., gender, race) or continuous covariates (e.g., age) exist that might influence the measurement properties of items; these variables are often correlated, making traditional tests that consider each variable separately less useful. Here, we propose the application of Bayesian Moderated Nonlinear Factor Analysis to overcome limitations of traditional approaches to detect DIF. We investigate how modern Bayesian shrinkage priors can be used to identify DIF items in situations with many groups and continuous covariates. We compare the performance of lasso-type, spike-and-slab, and global-local shrinkage priors (e.g., horseshoe) to standard normal and small variance priors. Results indicate that spike-and-slab and lasso priors outperform the other priors. Horseshoe priors provide slightly lower power compared to lasso and spike-and-slab priors. Small variance priors result in very low power to detect DIF with sample sizes below 800, and normal priors may produce severely inflated type I error rates. We illustrate the approach with data from the PISA 2018 study. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":"482-512"},"PeriodicalIF":7.6,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9595041","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Psychological methodsPub Date : 2025-06-01Epub Date: 2023-05-25DOI: 10.1037/met0000575
Sophie W Berkhout, Noémi K Schuurman, Ellen L Hamaker
{"title":"A tool to simulate and visualize dyadic interaction dynamics.","authors":"Sophie W Berkhout, Noémi K Schuurman, Ellen L Hamaker","doi":"10.1037/met0000575","DOIUrl":"10.1037/met0000575","url":null,"abstract":"<p><p>ynamic models are becoming increasingly popular to study the dynamic processes of dyadic interactions. In this article, we present a Dyadic Interaction Dynamics (DID) Shiny app which provides simulations and visualizations of data from several models that have been proposed for the analysis of dyadic data. We propose data generation as a tool to inspire and guide theory development and elaborate on how to connect substantive ideas to specific features of these models. We begin by discussing the basics of dynamic models with dyadic interactions. Then we present several models and illustrate model-implied behavior through generated data, accompanied by the DID Shiny app which allows researchers to generate and visualize their own data. Specifically, we consider: (a) the first-order vector autoregressive (VAR(1)) model; (b) the latent VAR(1) model; (c) the time-varying VAR(1) model; (d) the threshold VAR(1) model; (e) the hidden Markov model; and (f) the Markov-switching VAR(1) model. Finally, we demonstrate these models using empirical examples. We aim to give researchers more insight into what dynamic modeling approach fits their research question and data best. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":"599-621"},"PeriodicalIF":7.6,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9876145","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Aggregating evidence for a central theory from diverse studies using the generalized order-restricted information criterion.","authors":"R M Kuiper, Eli-Boaz Clapper","doi":"10.1037/met0000755","DOIUrl":"https://doi.org/10.1037/met0000755","url":null,"abstract":"<p><p>In social and behavioral science, the gold standard for scientific evidence is finding results that are consistent across independent studies. To summarize results from multiple studies, parameter estimates are conventionally aggregated with meta-analysis. However, having comparable estimates is limited to studies that share the same context and design, which often means that a wealth of information remains unexploited. This article proposes generalized order-restricted information criterion (approximation) (GORIC[A]) evidence aggregation: an alternative and/or complementary statistical tool for the aggregation of evidence across studies. Rather than aggregating parameter estimates to come to an overall estimate, GORIC(A) evidence aggregation combines relative support for a shared central theory (i.e., evidence) and quantifies the overall relative support. It does so using GORIC(A), an information criterion that can evaluate both equality and inequality/order restrictions. GORIC(A) can be applied to a single study, and this GORIC(A) evidence can be aggregated over multiple studies, irrespective of how the research is conducted. The method is validated with a simulation study that shows that GORIC(A) evidence aggregation is not affected by study-design heterogeneity and can be used for evidence synthesis. This implies that GORIC(A) evidence aggregation can successfully combine relative support for a central theory over a widely diverse set of studies. This increases the available information to investigate a theory. Furthermore, GORIC(A) evidence aggregation aids in robustness and confidence of results because it can take into account the results of all type of studies that (can) examine the central theory. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-05-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144182508","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"When predictors sum to a constant: Trade-off effect analysis using a regression model based on isometric log-ratio transformation.","authors":"Jieyuan Dong, Hongyun Liu","doi":"10.1037/met0000668","DOIUrl":"https://doi.org/10.1037/met0000668","url":null,"abstract":"<p><p>The standard regression model is not feasible when the sum of predictors is a constant, which is a common occurrence in proportional data or ipsative data. Davison et al. (2022) described a set of reduced-rank regression models in which each regression coefficient can be interpreted as a predictor trade-off effect. However, the assumption of linearity and symmetry in their method is too rigid, and the compositional nature of the predictors should not be disregarded. In this article, from the perspective of compositional data, a new method named isometric-log-ratio-transformed trade-off effect analysis (ITEA) is proposed. The predictors are transformed into isometric log-ratio coordinates using a planned sequential binary partition, and trade-off effects are then estimated using a regression model with isometric log-ratio coordinates. Instead of directly relying on regression coefficients, the trade-off effect is defined as the difference in the dependent variable before and after the trade-off, from which the 95% confidence interval can be further derived. Moreover, the main results of the ITEA are not affected by the variation in orthonormal bases. Applying the ITEA to the data in Davison et al.'s (2022) study yields more flexible and interpretable results of trade-off effects. We also provide an empirical example of a forced-choice questionnaire to verify the validity of the ITEA, with visualization attempts of trade-off effects under different conditions. Usefulness, suitable applications, and potential extensions are discussed. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-05-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144180774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sophie W Berkhout, Noémi K Schuurman, Ellen L Hamaker
{"title":"Let sleeping dogs lie? How to deal with the night gap problem in experience sampling method data.","authors":"Sophie W Berkhout, Noémi K Schuurman, Ellen L Hamaker","doi":"10.1037/met0000762","DOIUrl":"https://doi.org/10.1037/met0000762","url":null,"abstract":"<p><p>Night gaps are inherent to data obtained with the experience sampling method (ESM). When such data are used to study lagged relations between variables-such as autoregression within the same variable, and cross-lagged regressions between different variables-the actual role of night gaps is typically not investigated. However, there are various methods to handle them in analyses. Common solutions involve (a) ignoring the night gap by considering the night interval as a regular interval; (b) removing the night gap by not regressing the first measurement of the day on the last measurement of the previous day; or (c) treating the night gap as a missing data problem. The goal of this article is to make explicit the theoretical implications of these three methods within the context of the first-order autoregressive model. Additionally, we propose an alternative modeling approach that allows us to study the implications of the night gap in more detail. Moreover, given that the current methods are special cases of the proposed alternative, we can test which method best describes the process of interest. Through an empirical <i>N</i> = 1 example with various ESM variables, we demonstrate that the best-fitting method differs per variable. This implies that some processes may exhibit different dynamics during the night than during the daytime, providing a stepping stone to understanding and modeling night gaps in ESM. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144128429","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Supplemental Material for A Bayesian Classifier for Fractal Characterization of Short Behavioral Series","authors":"","doi":"10.1037/met0000562.supp","DOIUrl":"https://doi.org/10.1037/met0000562.supp","url":null,"abstract":"","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":"34 1","pages":""},"PeriodicalIF":7.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144229357","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reliability for multilevel data: A correlation approach.","authors":"Tzu-Yao Lin, Francis Tuerlinckx, Sophie Vanbelle","doi":"10.1037/met0000738","DOIUrl":"https://doi.org/10.1037/met0000738","url":null,"abstract":"<p><p>Studying the reliability of a measurement instrument is essential. Despite the recognition of the importance of reliability in psychology and medicine and the various reliability coefficients that have been proposed, research on reliability for nested or multilevel data, ubiquitously in observational studies, remains limited. Two recent articles (Schönbrodt et al., 2022; ten Hove et al., 2022) address how to quantify reliability in multilevel settings based on generalizability theory. Specifically, ten Hove et al. (2022) defined between-cluster and within-cluster interrater intraclass correlation coefficients for multilevel designs where persons or raters are nested within clusters. Schönbrodt et al. (2022) also defined reliability coefficients at between-cluster and within-cluster (i.e., between-person) levels for designs where persons nested in couples are assessed numerous times daily over a number of days. Nevertheless, when applied to a common design, both approaches give inconsistent results regarding their definition of cluster-level reliability. In this article, we propose an alternative approach to defining reliability coefficients for multilevel data that are based on calculating the expected correlation between repeated measurements. We will compare our approach with that of Schönbrodt et al. (2022) and ten Hove et al. (2022) and explain the differences between the three approaches in a number of common nested data structures: (a) raters crossed with both persons and clusters, but persons are nested within clusters, (b) raters nested within both persons and clusters, and (c) persons nested in clusters and crossed with raters and days. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144128431","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Timescale mismatch in intensive longitudinal data: Current issues and possible solutions based on dynamic structural equation models.","authors":"Xiaohui Luo,Yueqin Hu,Hongyun Liu","doi":"10.1037/met0000749","DOIUrl":"https://doi.org/10.1037/met0000749","url":null,"abstract":"Intensive longitudinal data have been increasingly used to examine dynamic bidirectional relations between variables. However, the problem of timescale mismatch between variables faced by applied researchers remains understudied. Under the dynamic structural equation modeling framework, previous studies used the partial-path model and the average-score model, respectively, to explore the dynamic interaction processes and overall reciprocal effects between variables with mismatched timescales. The present study aimed to evaluate the performance of the existing modeling approaches and the effectiveness of the improved approaches (i.e., the full-path model, the factor model, and the adjusted factor model). Study 1 showed that the full-path model, which considered the cross-lagged effects of all time points of variables with denser timescales, better reflected dynamic interaction processes and time-specific effects between variables than the partial-path model. Study 2-1 found that the estimates of autoregressive and cross-lagged effects between timescale mismatched variables were biased in the average-score model, but accurate in the factor model. Study 2-2 further suggested that when there were regression effects between different time points of variables with denser timescales, the adjusted factor model obtained less bias than the factor model, yet the difference is negligible when the regression effects are small. Study 3 used empirical data with timescale mismatched variables to illustrate the differences of all modeling approaches. This study identified the important problem of timescale mismatch in intensive longitudinal data and its possible solutions, providing methodological guidance and valuable insights for data collection and analysis of variables with mismatched timescales. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":"10 1","pages":""},"PeriodicalIF":7.0,"publicationDate":"2025-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144087853","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mohammadhossein Manuel Haqiqatkhah,Ellen L Hamaker
{"title":"Daily dynamics and weekly rhythms: A tutorial on seasonal autoregressive-moving average models combined with day-of-the-week effects.","authors":"Mohammadhossein Manuel Haqiqatkhah,Ellen L Hamaker","doi":"10.1037/met0000756","DOIUrl":"https://doi.org/10.1037/met0000756","url":null,"abstract":"Daily diary data of emotional experiences are typically modeled with a first-order autoregressive model to account for possible day-to-day dynamics. However, our emotional experiences are likely influenced by the weekly rhythm of our activities, which may be reflected by (a) day-of-the-week effects (DOWEs), where different weekdays are characterized by different means; and (b) week-to-week dynamics, where weekday-specific activities and experiences have a delayed effect on the emotions that we experience on the same weekday a week later. While DOWEs have been studied occasionally, week-to-week dynamics have been largely ignored in psychological research. We present a set of complementary visualization techniques for detecting weekly rhythms and day-to-day dynamics in time series data. Subsequently, we introduce the family of seasonal autoregressive-moving average models from the econometrics literature, extend them with DOWEs models, and show how their components appear in visualizations. We then provide a tutorial on fitting these models in R, discuss model fit and model selection, and apply them to a daily diary dataset of 56-101 daily measures from 98 individuals. The results suggest that most individuals in the sample may be characterized by patterns and dynamics that the current practices in psychological research cannot capture adequately, and we discuss their implications for current psychological research practices. Reflecting critically on the limitations of our approach, we regard our findings as an initial step to encourage researchers to move beyond the ubiquitous paradigm of lag-1 autoregressive modeling and consider other types of dynamics at different timescales, and put forth ways forward. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":"32 1","pages":""},"PeriodicalIF":7.0,"publicationDate":"2025-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144087848","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Understanding measurement precision from a regression perspective.","authors":"Yang Liu,Jolynn Pek,Alberto Maydeu-Olivares","doi":"10.1037/met0000763","DOIUrl":"https://doi.org/10.1037/met0000763","url":null,"abstract":"We adopt and expand McDonald's (2011) regression framework for measurement precision, integrating two key perspectives: (a) reliability of observed scores and (b) optimal prediction of latent scores. Reliability arises from a measurement decomposition of an observed score into its true score and measurement error. In contrast, proportional reduction in mean squared error (PRMSE) arises from a prediction decomposition of a latent score into its optimal predictor (the observed expected a posteriori [EAP] score) and prediction error. Reliability is the coefficient of determination obtained by two isomorphic regressions: regressing the observed score on its true score or on all the latent variables. Similarly, PRMSE is the coefficient of determination obtained from two isomorphic regressions: regressing the latent score on its observed EAP score or all the manifest variables. A key implication of this regression framework is that both reliability and PRMSE can be estimated using a Monte Carlo (MC) method, which is particularly useful when no analytic formula is available or when the analytic calculation is involved. We illustrate these concepts with a factor analysis model and a two-parameter logistic model, in which we compute reliability coefficients for different observed scores and PRMSE for different latent scores. Additionally, we provide a numerical example demonstrating how the MC method can be used to estimate reliability and PRMSE within a two-dimensional item response tree model. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":"44 1","pages":""},"PeriodicalIF":7.0,"publicationDate":"2025-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144087847","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}