Psychological methods最新文献

A tutorial on using generative models to advance psychological science: Lessons from the reliability paradox. 使用生成模型推进心理科学的教程：可靠性悖论的启示。

IF 7 1区心理学

Psychological methods Pub Date : 2025-04-14 DOI: 10.1037/met0000674

Nathaniel Haines,Peter D Kvam,Louis Irving,Colin Tucker Smith,Theodore P Beauchaine,Mark A Pitt,Woo-Young Ahn,Brandon M Turner

{"title":"A tutorial on using generative models to advance psychological science: Lessons from the reliability paradox.","authors":"Nathaniel Haines,Peter D Kvam,Louis Irving,Colin Tucker Smith,Theodore P Beauchaine,Mark A Pitt,Woo-Young Ahn,Brandon M Turner","doi":"10.1037/met0000674","DOIUrl":"https://doi.org/10.1037/met0000674","url":null,"abstract":"Theories of individual differences are foundational to psychological and brain sciences, yet they are traditionally developed and tested using superficial summaries of data (e.g., mean response times) that are disconnected from our otherwise rich conceptual theories of behavior. To resolve this theory-description gap, we review the generative modeling approach, which involves formally specifying how behavior is generated within individuals, and in turn how generative mechanisms vary across individuals. Generative modeling shifts our focus away from estimating descriptive statistical \"effects\" toward estimating psychologically interpretable parameters, while simultaneously enhancing the reliability and validity of our measures. We demonstrate the utility of generative modeling in the context of the \"reliability paradox,\" a phenomenon wherein replicable group effects (e.g., Stroop effect) fail to capture individual differences (e.g., low test-retest reliability). Simulations and empirical data from the Implicit Association Test and Stroop, Flanker, Posner, and delay discounting tasks show that generative models yield (a) more theoretically informative parameters, and (b) higher test-retest reliability estimates relative to traditional approaches, illustrating their potential for enhancing theory development. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":"108 1","pages":""},"PeriodicalIF":7.0,"publicationDate":"2025-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143836590","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The use of large language models for qualitative research: The Deep Computational Text Analyser (DECOTA). 在定性研究中使用大型语言模型：深度计算文本分析器（DECOTA）。

IF 7.6 1区心理学

Psychological methods Pub Date : 2025-04-07 DOI: 10.1037/met0000753

Lois Player, Ryan Hughes, Kaloyan Mitev, Lorraine Whitmarsh, Christina Demski, Nicholas Nash, Trisevgeni Papakonstantinou, Mark Wilson

{"title":"The use of large language models for qualitative research: The Deep Computational Text Analyser (DECOTA).","authors":"Lois Player, Ryan Hughes, Kaloyan Mitev, Lorraine Whitmarsh, Christina Demski, Nicholas Nash, Trisevgeni Papakonstantinou, Mark Wilson","doi":"10.1037/met0000753","DOIUrl":"https://doi.org/10.1037/met0000753","url":null,"abstract":"Machine-assisted approaches for free-text analysis are rising in popularity, owing to a growing need to rapidly analyze large volumes of qualitative data. In both research and policy settings, these approaches have promise in providing timely insights into public perceptions and enabling policymakers to understand their community's needs. However, current approaches still require expert human interpretation-posing a financial and practical barrier for those outside of academia. For the first time, we propose and validate the Deep Computational Text Analyser (DECOTA)-a novel machine learning methodology that automatically analyzes large free-text data sets and outputs concise themes. Building on structural topic modeling approaches, we used two fine-tuned large language models and sentence transformers to automatically derive \"codes\" and their corresponding \"themes\", as in inductive thematic analysis. To fully automate the process, we designed and validated a novel algorithm to choose the optimal number of \"topics\" for the structural topic modeling. DECOTA outputs key codes and themes, their prevalence, and how prevalence varies across covariates such as age and gender. Each code is accompanied by three representative quotes. Four data sets previously analyzed using thematic analysis were triangulated with DECOTA's codes and themes. We found that DECOTA is approximately 378 times faster and 1,920 times cheaper than human coding and consistently yields codes in agreement with or complementary to human coding (averaging 91.6% for codes and 90% for themes). The implications for evidence-based policy development, public engagement with policymaking, and psychometric measure development are discussed. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143803939","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Zero inflation in intensive longitudinal data: Why is it important and how should we deal with it? 密集纵向数据中的零通胀：为什么它很重要，我们应该如何应对？

IF 7.6 1区心理学

Psychological methods Pub Date : 2025-04-07 DOI: 10.1037/met0000754

Sijing S J Shao, Ziqian Xu, Qimin Liu, Kenneth McClure, Ross Jacobucci, Scott E Maxwell, Zhiyong Zhang

{"title":"Zero inflation in intensive longitudinal data: Why is it important and how should we deal with it?","authors":"Sijing S J Shao, Ziqian Xu, Qimin Liu, Kenneth McClure, Ross Jacobucci, Scott E Maxwell, Zhiyong Zhang","doi":"10.1037/met0000754","DOIUrl":"https://doi.org/10.1037/met0000754","url":null,"abstract":"This study addresses the challenge of analyzing intensive longitudinal data (ILD) with zero-inflated autoregressive processes. ILD, characterized by intensive longitudinal measurements, often exhibit excessive zeros and temporal dependencies. Neglecting zero inflation or mishandling it can lead to biased parameter estimates and inaccurate conclusions. To overcome this issue, we propose a novel zero-inflated process change multilevel autoregressive (ZIP-CAR) model that incorporates zero inflation using a Bayesian framework. We compare the performance of the proposed method with existing methods through a simulation study and demonstrate its advantages in accurately estimating parameters and improving statistical power. Additionally, we apply the ZIP-CAR model to a real intensive longitudinal data set on problematic drinking behaviors, highlighting its effectiveness in capturing autoregressive and cross-lag effects while accounting for zero inflation. The results underscore the importance of addressing zero inflation in ILD analysis and provide practical recommendations for researchers. Our proposed model offers a valuable tool for analyzing ILD with zero-inflated autoregressive processes, facilitating a more comprehensive understanding of dynamic behavioral changes over time. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143804006","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Bayesian nonparametric latent class analysis with different item types. 不同项目类型的贝叶斯非参数潜在类分析。

IF 7.6 1区心理学

Psychological methods Pub Date : 2025-03-20 DOI: 10.1037/met0000728

Meng Qiu, Sally Paganin, Ilsang Ohn, Lizhen Lin

{"title":"Bayesian nonparametric latent class analysis with different item types.","authors":"Meng Qiu, Sally Paganin, Ilsang Ohn, Lizhen Lin","doi":"10.1037/met0000728","DOIUrl":"https://doi.org/10.1037/met0000728","url":null,"abstract":"Latent class analysis (LCA) requires deciding on the number of classes. This is traditionally addressed by fitting several models with an increasing number of classes and determining the optimal one using model selection criteria. However, different criteria can suggest different models, making it difficult to reach a consensus on the best criterion. Bayesian nonparametric LCA based on the Dirichlet process mixture (DPM) model is a flexible alternative approach that allows for inferring the number of classes from the data. In this article, we introduce a DPM-based mixed-mode LCA model, referred to as DPM-MMLCA, which clusters individuals based on indicators measured on mixed metrics. We illustrate two algorithms for posterior estimation and discuss inferential procedures to estimate the number of classes and their composition. A simulation study is conducted to compare the performance of the DPM-MMLCA with the traditional mixed-mode LCA under different scenarios. Five design factors are considered, including the number of latent classes, the number of observed variables, sample size, mixing proportions, and class separation. Performance measures include evaluating the correct identification of the number of latent classes, parameter recovery, and assignment of class labels. The Bayesian nonparametric LCA approach is illustrated using three real data examples. Additionally, a hands-on tutorial using R and the nimble package is provided for ease of implementation. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143670682","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Adjusting for nonrepresentativeness in continuous norming using multilevel regression and poststratification. 利用多水平回归和后分层调整连续归一化的非代表性。

IF 7.6 1区心理学

Psychological methods Pub Date : 2025-03-13 DOI: 10.1037/met0000752

Klazien de Vries, Marieke E Timmerman, Anja F Ernst, Casper J Albers

{"title":"Adjusting for nonrepresentativeness in continuous norming using multilevel regression and poststratification.","authors":"Klazien de Vries, Marieke E Timmerman, Anja F Ernst, Casper J Albers","doi":"10.1037/met0000752","DOIUrl":"https://doi.org/10.1037/met0000752","url":null,"abstract":"In psychological test norming, nonrepresentativeness in background variables in the normative sample can lead to bias in the normed score estimates. Because representativeness is difficult to establish in practice, adjustment methods are needed to combat this bias. As a candidate adjustment method, we investigated generalized additive models for location, scale, and shape with multilevel regression and poststratification (GAMLSS + MRP), the combination of MRP and continuous norming with GAMLSS. This adjustment method was then compared to current adjustment methods in continuous norming using weighted regression: GAMLSS + P (with poststratification) and cNORM + R (with raking). The results of our simulation showed that GAMLSS + MRP was generally more efficient than GAMLSS + P and cNORM + R. Furthermore, GAMLSS + MRP was better than the current methods at reducing bias in samples where the nonrepresentativeness was age-dependent. We argue that GAMLSS + MRP is a valid adjustment method in continuous norming and recommend this adjustment method to mitigate bias in nonrepresentative normative samples. To facilitate the use of GAMLSS + MRP in practice, we provide a step-wise approach for the implementation of GAMLSS + MRP. We illustrate this approach by deriving normed scores from the normative data of the third Schlichting language test. All analysis code for this illustration is provided. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143625380","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Unsupervised [randomly responding] survey bot detection: In search of high classification accuracy. 无监督[随机响应]调查机器人检测：寻求高分类精度。

IF 7.6 1区心理学

Psychological methods Pub Date : 2025-03-10 DOI: 10.1037/met0000746

Carl F Falk, Amaris Huang, Michael John Ilagan

{"title":"Unsupervised [randomly responding] survey bot detection: In search of high classification accuracy.","authors":"Carl F Falk, Amaris Huang, Michael John Ilagan","doi":"10.1037/met0000746","DOIUrl":"https://doi.org/10.1037/met0000746","url":null,"abstract":"While online survey data collection has become popular in the social sciences, there is a risk of data contamination by computer-generated random responses (i.e., bots). Bot prevalence poses a significant threat to data quality. If deterrence efforts fail or were not set up in advance, researchers can still attempt to detect bots already present in the data. In this research, we study a recently developed algorithm to detect survey bots. The algorithm requires neither a measurement model nor a sample of known humans and bots; thus, it is model agnostic and unsupervised. It involves a permutation test under the assumption that Likert-type items are exchangeable for bots, but not humans. While the algorithm maintains a desired sensitivity for detecting bots (e.g., 95%), its classification accuracy may depend on other inventory-specific or demographic factors. Generating hypothetical human responses from a well-known item response theory model, we use simulations to understand how classification accuracy is affected by item properties, the number of items, the number of latent factors, and factor correlations. In an additional study, we simulate bots to contaminate real human data from 35 publicly available data sets to understand the algorithm's classification accuracy under a variety of real measurement instruments. Through this work, we identify conditions under which classification accuracy is around 95% or above, but also conditions under which accuracy is quite low. In brief, performance is better with more items, more categories per item, and a variety in the difficulty or means of the survey items. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143597819","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A tutorial on estimating dynamic treatment regimes from observational longitudinal data using lavaan.

IF 7.6 1区心理学

Psychological methods Pub Date : 2025-03-06 DOI: 10.1037/met0000748

Wen Wei Loh, Terrence D Jorgensen

{"title":"A tutorial on estimating dynamic treatment regimes from observational longitudinal data using lavaan.","authors":"Wen Wei Loh, Terrence D Jorgensen","doi":"10.1037/met0000748","DOIUrl":"https://doi.org/10.1037/met0000748","url":null,"abstract":"Psychological and behavioral scientists develop interventions toward addressing pressing societal challenges. But such endeavors are complicated by treatments that change over time as individuals' needs and responses evolve. For instance, students initially in a multiyear mentoring program to improve future academic outcomes may not continue with the program after interim school engagement improves. Conventional interventions bound by rigid treatment assignments cannot adapt to such time-dependent heterogeneity, thus undermining the interventions' practical relevance and leading to inefficient implementations. Dynamic treatment regimes (DTRs) are a class of interventions that are more tailored, relevant, and efficient than conventional interventions. DTRs, an established approach in the causal inference and personalized medicine literature, are designed to address the causal query: how can individual treatment assignments in successive time points be adapted, based on time-evolving responses, to optimize the intervention's effectiveness? This tutorial offers an accessible introduction to DTRs using a simple example from the psychology literature. We describe how, using observational data from a single naturally occurring longitudinal study, to estimate the outcomes had different DTRs been counterfactually implemented. To improve accessibility, we implement the estimation procedure in lavaan, a freely available statistical software popular in psychology and social science research. We hope this tutorial guides researchers on framing, interpreting, and testing DTRs in their investigations. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143567974","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Network science in psychology. 心理学中的网络科学。

IF 7.6 1区心理学

Psychological methods Pub Date : 2025-03-03 DOI: 10.1037/met0000745

Tracy Sweet, Selena Wang

引用次数: 0

Iterated community detection in psychological networks.

IF 7.6 1区心理学

Psychological methods Pub Date : 2025-03-03 DOI: 10.1037/met0000744

M A Werner, J de Ron, E I Fried, D J Robinaugh

{"title":"Iterated community detection in psychological networks.","authors":"M A Werner, J de Ron, E I Fried, D J Robinaugh","doi":"10.1037/met0000744","DOIUrl":"https://doi.org/10.1037/met0000744","url":null,"abstract":"Psychological network models often feature communities: subsets of nodes that are more densely connected to themselves than to other nodes. The Spinglass algorithm is a popular method of detecting communities within a network, but it is a nondeterministic algorithm, meaning that the results can vary from one iteration to the next. There is no established method for determining the optimal solution or for evaluating instability across iterations in the emerging discipline of network psychometrics. We addressed this need by introducing and evaluating iterated community detection: Spinglass (IComDetSpin), a method for aggregating across multiple Spinglass iterations to identify the most frequent solution and quantify and visualize the instability of the solution across iterations. In two simulation studies, we evaluated (a) the performance of IComDetSpin in identifying the true community structure and (b) information about the fuzziness of community boundaries; information that is not available with a single iteration of Spinglass. In Study 1, IComDetSpin outperformed single-iteration Spinglass in identifying the true number of communities and performed comparably to Walktrap. In Study 2, we extended our evaluation to networks estimated from simulated data and found that both IComDetSpin and Exploratory Graph Analysis (a well-established community detection method in network psychometrics) performed well and that IComDetSpin outperformed Exploratory Graph Analysis when correlations between communities were high and number of nodes per community was lower (5 vs. 10). Overall, IComDetSpin improved the performance of Spinglass and provided unique information about the stability of community detection results and fuzziness in community structure. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143543201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Unidim: An index of scale homogeneity and unidimensionality. Unidim：量表同质性和单维性指数。

IF 7.6 1区心理学

Psychological methods Pub Date : 2025-03-03 DOI: 10.1037/met0000729

William Revelle, David Condon

{"title":"Unidim: An index of scale homogeneity and unidimensionality.","authors":"William Revelle, David Condon","doi":"10.1037/met0000729","DOIUrl":"https://doi.org/10.1037/met0000729","url":null,"abstract":"How to evaluate how well a psychological scale measures just one construct is a recurring problem in assessment. We introduce an index, u, of the unidimensionality and homogeneity of a scale. u is just the product of two other indices: τ (a measure of τ equivalence) and ρc (a measure of congeneric fit). By combining these two indices into one, we provide a simple index of the unidimensionality and homogeneity of a scale. We evaluate u through simulations and with real data sets. Simulations of u across one-factor scales ranging from three to 24 items with various levels of factor homogeneity show that τ and, therefore, u are sensitive to the degree of factor homogeneity. Additional tests with multifactorial scales representing 9, 18, 27, and 36 items with a hierarchical factor structure varying in a general factor loading show that ρc and, therefore, u are sensitive to the general factor saturation of a test. We also demonstrate the performance of u on 45 different publicly available personality and ability measures. Comparisons with traditional measures (i.e., ωh, α, ωt, comparative fit index, and explained common variance) show that u has greater sensitivity to unidimensional structure and less sensitivity to the number of items in a scale. u is easily calculated with open source statistical packages and is relatively robust to sample sizes ranging from 100 to 5,000. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":""},"PeriodicalIF":7.6,"publicationDate":"2025-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143543205","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0