Lasse Elsemüller, Martin Schnuerch, Paul-Christian Bürkner, Stefan T Radev
{"title":"A deep learning method for comparing Bayesian hierarchical models.","authors":"Lasse Elsemüller, Martin Schnuerch, Paul-Christian Bürkner, Stefan T Radev","doi":"10.1037/met0000645","DOIUrl":"https://doi.org/10.1037/met0000645","url":null,"abstract":"<p><p>Bayesian model comparison (BMC) offers a principled approach to assessing the relative merits of competing computational models and propagating uncertainty into model selection decisions. However, BMC is often intractable for the popular class of hierarchical models due to their high-dimensional nested parameter structure. To address this intractability, we propose a deep learning method for performing BMC on any set of hierarchical models which can be instantiated as probabilistic programs. Since our method enables amortized inference, it allows efficient re-estimation of posterior model probabilities and fast performance validation prior to any real-data application. In a series of extensive validation studies, we benchmark the performance of our method against the state-of-the-art bridge sampling method and demonstrate excellent amortized inference across all BMC settings. We then showcase our method by comparing four hierarchical evidence accumulation models that have previously been deemed intractable for BMC due to partly implicit likelihoods. Additionally, we demonstrate how transfer learning can be leveraged to enhance training efficiency. We provide reproducible code for all analyses and an open-source implementation of our method. (PsycInfo Database Record (c) 2024 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":null,"pages":null},"PeriodicalIF":7.0,"publicationDate":"2024-05-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140857735","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Are factor scores measurement invariant?","authors":"Mark H C Lai, Winnie W-Y Tse","doi":"10.1037/met0000658","DOIUrl":"https://doi.org/10.1037/met0000658","url":null,"abstract":"<p><p>There has been increased interest in practical methods for integrative analysis of data from multiple studies or samples, and using factor scores to represent constructs has become a popular and practical alternative to latent variable models with all individual items. Although researchers are aware that scores representing the same construct should be on a similar metric across samples-namely they should be measurement invariant-for integrative data analysis, the methodological literature is unclear whether factor scores would satisfy such a requirement. In this note, we show that even when researchers successfully calibrate the latent factors to the same metric across samples, factor scores-which are estimates of the latent factors but not the factors themselves-may not be measurement invariant. Specifically, we prove that factor scores computed based on the popular regression method are generally not measurement invariant. Surprisingly, such scores can be noninvariant even when the items are invariant. We also demonstrate that our conclusions generalize to similar shrinkage scores in item response models for discrete items, namely the expected a posteriori scores and the maximum a posteriori scores. Researchers should be cautious in directly using factor scores for cross-sample analyses, even when such scores are obtained from measurement models that account for noninvariance. (PsycInfo Database Record (c) 2024 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":null,"pages":null},"PeriodicalIF":7.0,"publicationDate":"2024-05-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140857844","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Supplemental Material for Can Cross-Lagged Panel Modeling Be Relied on to Establish Cross-Lagged Effects? The Case of Contemporaneous and Reciprocal Effects","authors":"","doi":"10.1037/met0000661.supp","DOIUrl":"https://doi.org/10.1037/met0000661.supp","url":null,"abstract":"","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":null,"pages":null},"PeriodicalIF":7.0,"publicationDate":"2024-05-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141019755","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Supplemental Material for Are Factor Scores Measurement Invariant?","authors":"","doi":"10.1037/met0000658.supp","DOIUrl":"https://doi.org/10.1037/met0000658.supp","url":null,"abstract":"","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":null,"pages":null},"PeriodicalIF":7.0,"publicationDate":"2024-05-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141021088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Supplemental Material for A Deep Learning Method for Comparing Bayesian Hierarchical Models","authors":"","doi":"10.1037/met0000645.supp","DOIUrl":"https://doi.org/10.1037/met0000645.supp","url":null,"abstract":"","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":null,"pages":null},"PeriodicalIF":7.0,"publicationDate":"2024-05-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141019028","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Causal relationships in longitudinal observational data: An integrative modeling approach.","authors":"C. Biazoli, João R. Sato, Michael Pluess","doi":"10.1037/met0000648","DOIUrl":"https://doi.org/10.1037/met0000648","url":null,"abstract":"Much research in psychology relies on data from observational studies that traditionally do not allow for causal interpretation. However, a range of approaches in statistics and computational sciences have been developed to infer causality from correlational data. Based on conceptual and theoretical considerations on the integration of interventional and time-restrainment notions of causality, we set out to design and empirically test a new approach to identify potential causal factors in longitudinal correlational data. A principled and representative set of simulations and an illustrative application to identify early-life determinants of cognitive development in a large cohort study are presented. The simulation results illustrate the potential but also the limitations for discovering causal factors in observational data. In the illustrative application, plausible candidates for early-life determinants of cognitive abilities in 5-year-old children were identified. Based on these results, we discuss the possibilities of using exploratory causal discovery in psychological research but also highlight its limits and potential misuses and misinterpretations. (PsycInfo Database Record (c) 2024 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":null,"pages":null},"PeriodicalIF":7.0,"publicationDate":"2024-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140672431","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Combinational regularity analysis (CORA): An introduction for psychologists.","authors":"Alrik Thiem, L. Mkrtchyan, Zuzana Sebechlebská","doi":"10.1037/met0000653","DOIUrl":"https://doi.org/10.1037/met0000653","url":null,"abstract":"Increasingly, psychologists make use of modern configurational comparative methods (CCMs), such as qualitative comparative analysis (QCA) and coincidence analysis (CNA), to infer regularity-theoretic causal structures from psychological data. At the same time, existing CCMs remain unable to reveal such structures in the presence of complex effects. Given the strong emphasis configurational methodology generally puts on the notion of complex causation, and the ubiquity of multieffect problems in psychological research, such as multimorbidity and polypharmacy, this limitation is severe. In this article, we introduce psychologists to combinational regularity analysis (CORA)-a new member in the family of CCMs-with which regularity-theoretic causal structures that may include complex effects can be uncovered. To this end, CORA draws on algorithms originally developed in electrical engineering for the analysis of multioutput switching circuits, which regulate the behavior of electrical signals between a set of inputs and a set of outputs. After having situated CORA within the landscape of modern CCMs, we present its technical foundations. Subsequently, we demonstrate the method's analytical and graphical capabilities by means of artificial and empirical data. To facilitate familiarization, we use the concept of the \"method game\" to compare CORA with QCA and CNA. Through CORA, configurational analyses of complex effects come into the analytical reach of CCMs. CORA thus represents a useful addition to the methodological toolkit of psychologists who want to analyze their data from a configurational perspective. (PsycInfo Database Record (c) 2024 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":null,"pages":null},"PeriodicalIF":7.0,"publicationDate":"2024-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140675744","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The plausibility of alternative data-generating mechanisms: Comment on and attempt at replication of Dishop (2022).","authors":"Jonas W. B. Lang, P. Bliese","doi":"10.1037/met0000650","DOIUrl":"https://doi.org/10.1037/met0000650","url":null,"abstract":"Dishop (see record 2022-78260-001) identifies the consensus emergence model (CEM) as a useful tool for future research on emergence but argues that autoregressive models with positive autoregressive effects are an important alternative data-generating mechanism that researchers need to rule out. Here, we acknowledge that alternative data-generating mechanisms are possibility for most, if not all, nonexperimental designs and appreciate Dishop's attempts to identify cases where the CEM could provide misleading results. However, in a series of independent simulations, we were unable to replicate two of three key analyses, and the results for the third analysis did not support the earlier conclusions. The discrepancies appear to originate from Dishop's simulation code and what appear to be inconsistent model specifications that neither simulate the models described in the article nor include notable positive autoregressive effects. We contribute to the wider literature by suggesting four key criteria that researchers can apply to evaluate the possibility of alternative data-generating mechanisms: Theory, parameter recovery, fit to real data, and context. Applied to autoregressive effects and emergence data, these criteria reveal that (a) theory in psychology would generally suggest negative instead of positive autoregressive effects for behavior, (b) it is challenging to recover true autoregressive parameters from simulated data, and (c) that real data sets across a number of different contexts show little to no evidence for autoregressive effects. Instead, our analyses suggest that CEM results are congruent with the temporal changes occurring within groups and that autoregressive effects do not lead to spurious CEM results. (PsycInfo Database Record (c) 2024 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":null,"pages":null},"PeriodicalIF":7.0,"publicationDate":"2024-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140676950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Supplemental Material for Causal Relationships in Longitudinal Observational Data: An Integrative Modeling Approach","authors":"","doi":"10.1037/met0000648.supp","DOIUrl":"https://doi.org/10.1037/met0000648.supp","url":null,"abstract":"","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":null,"pages":null},"PeriodicalIF":7.0,"publicationDate":"2024-04-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140689521","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ria H A Hoekstra, S. Epskamp, Andrew A. Nierenberg, D. Borsboom, Richard J. McNally
{"title":"Testing similarity in longitudinal networks: The Individual Network Invariance Test.","authors":"Ria H A Hoekstra, S. Epskamp, Andrew A. Nierenberg, D. Borsboom, Richard J. McNally","doi":"10.1037/met0000638","DOIUrl":"https://doi.org/10.1037/met0000638","url":null,"abstract":"The comparison of idiographic network structures to determine the presence of heterogeneity is a challenging endeavor in many applied settings. Previously, researchers eyeballed idiographic networks, computed correlations, and used techniques that make use of the multilevel structure of the data (e.g., group iterative multiple model estimation and multilevel vector autoregressive) to investigate individual differences. However, these methods do not allow for testing the (in)equality of idiographic network structures directly. In this article, we propose the Individual Network Invariance Test (INIT), which we implemented in the R package INIT. INIT extends common model comparison practices in structural equation modeling to idiographic network structures to test for (in)equality between idiographic networks. In a simulation study, we evaluated the performance of INIT on both saturated and pruned idiographic network structures by inspecting the rejection rate of the χ² difference test and model selection criteria, such as the Akaike information criterion (AIC) and Bayesian information criterion (BIC). Results show INIT performs adequately when t = 100 per individual. When applying INIT on saturated networks, the AIC performed best as a model selection criterion, while the BIC showed better results when applying INIT on pruned networks. In an empirical example, we highlight the possibilities of this new technique, illustrating how INIT provides researchers with a means of testing for (in)equality between idiographic network structures and within idiographic network structures over time. To conclude, recommendations for empirical researchers are provided. (PsycInfo Database Record (c) 2024 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":null,"pages":null},"PeriodicalIF":7.0,"publicationDate":"2024-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140714460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}