PsychometrikaPub Date : 2023-09-01Epub Date: 2023-05-29DOI: 10.1007/s11336-023-09921-w
Steffen Nestler, Edgar Erdfelder
{"title":"Random Effects Multinomial Processing Tree Models: A Maximum Likelihood Approach.","authors":"Steffen Nestler, Edgar Erdfelder","doi":"10.1007/s11336-023-09921-w","DOIUrl":"10.1007/s11336-023-09921-w","url":null,"abstract":"<p><p>The present article proposes and evaluates marginal maximum likelihood (ML) estimation methods for hierarchical multinomial processing tree (MPT) models with random and fixed effects. We assume that an identifiable MPT model with S parameters holds for each participant. Of these S parameters, R parameters are assumed to vary randomly between participants, and the remaining [Formula: see text] parameters are assumed to be fixed. We also propose an extended version of the model that includes effects of covariates on MPT model parameters. Because the likelihood functions of both versions of the model are too complex to be tractable, we propose three numerical methods to approximate the integrals that occur in the likelihood function, namely, the Laplace approximation (LA), adaptive Gauss-Hermite quadrature (AGHQ), and Quasi Monte Carlo (QMC) integration. We compare these three methods in a simulation study and show that AGHQ performs well in terms of both bias and coverage rate. QMC also performs well but the number of responses per participant must be sufficiently large. In contrast, LA fails quite often due to undefined standard errors. We also suggest ML-based methods to test the goodness of fit and to compare models taking model complexity into account. The article closes with an illustrative empirical application and an outlook on possible extensions and future applications of the proposed ML approach.</p>","PeriodicalId":54534,"journal":{"name":"Psychometrika","volume":"88 3","pages":"809-829"},"PeriodicalIF":2.9,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10444666/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10644937","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PsychometrikaPub Date : 2023-09-01Epub Date: 2023-06-01DOI: 10.1007/s11336-023-09917-6
Sayed H Kadhem, Aristidis K Nikoloulopoulos
{"title":"Factor Tree Copula Models for Item Response Data.","authors":"Sayed H Kadhem, Aristidis K Nikoloulopoulos","doi":"10.1007/s11336-023-09917-6","DOIUrl":"10.1007/s11336-023-09917-6","url":null,"abstract":"<p><p>Factor copula models for item response data are more interpretable and fit better than (truncated) vine copula models when dependence can be explained through latent variables, but are not robust to violations of conditional independence. To circumvent these issues, truncated vines and factor copula models for item response data are joined to define a combined model, the so-called factor tree copula model, with individual benefits from each of the two approaches. Rather than adding factors and causing computational problems and difficulties in interpretation and identification, a truncated vine structure is assumed on the residuals conditional on one or two latent variables. This structure can be better explained as a conditional dependence given a few interpretable latent variables. On the one hand, the parsimonious feature of factor models remains intact and any residual dependencies are being taken into account on the other. We discuss estimation along with model selection. In particular, we propose model selection algorithms to choose a plausible factor tree copula model to capture the (residual) dependencies among the item responses. Our general methodology is demonstrated with an extensive simulation study and illustrated by analyzing Post-Traumatic Stress Disorder.</p>","PeriodicalId":54534,"journal":{"name":"Psychometrika","volume":"88 3","pages":"776-802"},"PeriodicalIF":2.9,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10444667/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10644945","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PsychometrikaPub Date : 2023-09-01Epub Date: 2023-05-23DOI: 10.1007/s11336-023-09914-9
Sierra A Bainter, Thomas G McCauley, Mahmoud M Fahmy, Zachary T Goodman, Lauren B Kupis, J Sunil Rao
{"title":"Comparing Bayesian Variable Selection to Lasso Approaches for Applications in Psychology.","authors":"Sierra A Bainter, Thomas G McCauley, Mahmoud M Fahmy, Zachary T Goodman, Lauren B Kupis, J Sunil Rao","doi":"10.1007/s11336-023-09914-9","DOIUrl":"10.1007/s11336-023-09914-9","url":null,"abstract":"<p><p>In the current paper, we review existing tools for solving variable selection problems in psychology. Modern regularization methods such as lasso regression have recently been introduced in the field and are incorporated into popular methodologies, such as network analysis. However, several recognized limitations of lasso regularization may limit its suitability for psychological research. In this paper, we compare the properties of lasso approaches used for variable selection to Bayesian variable selection approaches. In particular we highlight advantages of stochastic search variable selection (SSVS), that make it well suited for variable selection applications in psychology. We demonstrate these advantages and contrast SSVS with lasso type penalization in an application to predict depression symptoms in a large sample and an accompanying simulation study. We investigate the effects of sample size, effect size, and patterns of correlation among predictors on rates of correct and false inclusion and bias in the estimates. SSVS as investigated here is reasonably computationally efficient and powerful to detect moderate effects in small sample sizes (or small effects in moderate sample sizes), while protecting against false inclusion and without over-penalizing true effects. We recommend SSVS as a flexible framework that is well-suited for the field, discuss limitations, and suggest directions for future development.</p>","PeriodicalId":54534,"journal":{"name":"Psychometrika","volume":"88 3","pages":"1032-1055"},"PeriodicalIF":2.9,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10202760/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10278469","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PsychometrikaPub Date : 2023-09-01DOI: 10.1007/s11336-023-09906-9
Sun-Joo Cho, Sarah Brown-Schmidt, Paul De Boeck, Matthew Naveiras, Si On Yoon, Aaron Benjamin
{"title":"Incorporating Functional Response Time Effects into a Signal Detection Theory Model.","authors":"Sun-Joo Cho, Sarah Brown-Schmidt, Paul De Boeck, Matthew Naveiras, Si On Yoon, Aaron Benjamin","doi":"10.1007/s11336-023-09906-9","DOIUrl":"https://doi.org/10.1007/s11336-023-09906-9","url":null,"abstract":"<p><p>Signal detection theory (SDT; Tanner & Swets in Psychological Review 61:401-409, 1954) is a dominant modeling framework used for evaluating the accuracy of diagnostic systems that seek to distinguish signal from noise in psychology. Although the use of response time data in psychometric models has increased in recent years, the incorporation of response time data into SDT models remains a relatively underexplored approach to distinguishing signal from noise. Functional response time effects are hypothesized in SDT models, based on findings from other related psychometric models with response time data. In this study, an SDT model is extended to incorporate functional response time effects using smooth functions and to include all sources of variability in SDT model parameters across trials, participants, and items in the experimental data. The extended SDT model with smooth functions is formulated as a generalized linear mixed-effects model and implemented in the gamm4 R package. The extended model is illustrated using recognition memory data to understand how conversational language is remembered. Accuracy of parameter estimates and the importance of modeling variability in detecting the experimental condition effects and functional response time effects are shown in conditions similar to the empirical data set via a simulation study. In addition, the type 1 error rate of the test for a smooth function of response time is evaluated.</p>","PeriodicalId":54534,"journal":{"name":"Psychometrika","volume":"88 3","pages":"1056-1086"},"PeriodicalIF":3.0,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10281168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PsychometrikaPub Date : 2023-09-01Epub Date: 2023-06-16DOI: 10.1007/s11336-023-09913-w
Weicong Lyu, Daniel M Bolt
{"title":"Rejoinder to Commentaries on Lyu, Bolt and Westby's \"Exploring the Effects of Item Specific Factors in Sequential and IRTree Models\".","authors":"Weicong Lyu, Daniel M Bolt","doi":"10.1007/s11336-023-09913-w","DOIUrl":"10.1007/s11336-023-09913-w","url":null,"abstract":"<p><p>We respond to the commentaries on Lyu, Bolt and Westby's \"Exploring the effects of item specific factors in sequential and IRTree models.\" The commentaries raise important points that allow us to clarify our theoretical expectation for item specific factors in many educational and psychological test items. At the same time, we agree with the commentaries in acknowledging challenges associated with providing empirical evidence for their presence and reflect on strategies that might support their estimation. We maintain that the principal concern is the ambiguity item specific factors create in attempting to interpret or use the parameters beyond the first node.</p>","PeriodicalId":54534,"journal":{"name":"Psychometrika","volume":"88 3","pages":"1026-1031"},"PeriodicalIF":2.9,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10272278","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PsychometrikaPub Date : 2023-09-01Epub Date: 2023-06-16DOI: 10.1007/s11336-023-09912-x
Weicong Lyu, Daniel M Bolt, Samuel Westby
{"title":"Exploring the Effects of Item-Specific Factors in Sequential and IRTree Models.","authors":"Weicong Lyu, Daniel M Bolt, Samuel Westby","doi":"10.1007/s11336-023-09912-x","DOIUrl":"10.1007/s11336-023-09912-x","url":null,"abstract":"<p><p>Test items for which the item score reflects a sequential or IRTree modeling outcome are considered. For such items, we argue that item-specific factors, although not empirically measurable, will often be present across stages of the same item. In this paper, we present a conceptual model that incorporates such factors. We use the model to demonstrate how the varying conditional distributions of item-specific factors across stages become absorbed into the stage-specific item discrimination and difficulty parameters, creating ambiguity in the interpretations of item and person parameters beyond the first stage. We discuss implications in relation to various applications considered in the literature, including methodological studies of (1) repeated attempt items; (2) answer change/review, (3) on-demand item hints; (4) item skipping behavior; and (5) Likert scale items. Our own empirical applications, as well as several examples published in the literature, show patterns of violations of item parameter invariance across stages that are highly suggestive of item-specific factors. For applications using sequential or IRTree models as analytical models, or for which the resulting item score might be viewed as outcomes of such a process, we recommend (1) regular inspection of data or analytic results for empirical evidence (or theoretical expectations) of item-specific factors; and (2) sensitivity analyses to evaluate the implications of item-specific factors for the intended inferences or applications.</p>","PeriodicalId":54534,"journal":{"name":"Psychometrika","volume":"88 3","pages":"745-775"},"PeriodicalIF":2.9,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10279931","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PsychometrikaPub Date : 2023-09-01Epub Date: 2023-06-16DOI: 10.1007/s11336-023-09916-7
Thorsten Meiser, Fabiola Reiber
{"title":"Item-Specific Factors in IRTree Models: When They Matter and When They Don't.","authors":"Thorsten Meiser, Fabiola Reiber","doi":"10.1007/s11336-023-09916-7","DOIUrl":"10.1007/s11336-023-09916-7","url":null,"abstract":"<p><p>Lyu et al. (Psychometrika, 2023) demonstrated that item-specific factors can cause spurious effects on the structural parameters of IRTree models for multiple nested response processes per item. Here, we discuss some boundary conditions and argue that person selection effects on item parameters are not unique to item-specific factors and that the effects presented by Lyu et al. (Psychometrika, 2023) may not generalize to the family of IRTree models as a whole. We conclude with the recommendation that IRTree model specification should be guided by theoretical considerations, rather than driven by data, in order to avoid misinterpretations of parameter differences.</p>","PeriodicalId":54534,"journal":{"name":"Psychometrika","volume":"88 3","pages":"739-744"},"PeriodicalIF":2.9,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10444655/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10279936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PsychometrikaPub Date : 2023-09-01Epub Date: 2023-06-14DOI: 10.1007/s11336-023-09920-x
Inhan Kang, Minjeong Jeon, Ivailo Partchev
{"title":"A Latent Space Diffusion Item Response Theory Model to Explore Conditional Dependence between Responses and Response Times.","authors":"Inhan Kang, Minjeong Jeon, Ivailo Partchev","doi":"10.1007/s11336-023-09920-x","DOIUrl":"10.1007/s11336-023-09920-x","url":null,"abstract":"<p><p>Traditional measurement models assume that all item responses correlate with each other only through their underlying latent variables. This conditional independence assumption has been extended in joint models of responses and response times (RTs), implying that an item has the same item characteristics fors all respondents regardless of levels of latent ability/trait and speed. However, previous studies have shown that this assumption is violated in various types of tests and questionnaires and there are substantial interactions between respondents and items that cannot be captured by person- and item-effect parameters in psychometric models with the conditional independence assumption. To study the existence and potential cognitive sources of conditional dependence and utilize it to extract diagnostic information for respondents and items, we propose a diffusion item response theory model integrated with the latent space of variations in information processing rate of within-individual measurement processes. Respondents and items are mapped onto the latent space, and their distances represent conditional dependence and unexplained interactions. We provide three empirical applications to illustrate (1) how to use an estimated latent space to inform conditional dependence and its relation to person and item measures, (2) how to derive diagnostic feedback personalized for respondents, and (3) how to validate estimated results with an external measure. We also provide a simulation study to support that the proposed approach can accurately recover its parameters and detect conditional dependence underlying data.</p>","PeriodicalId":54534,"journal":{"name":"Psychometrika","volume":"88 3","pages":"830-864"},"PeriodicalIF":2.9,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10332140","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PsychometrikaPub Date : 2023-09-01Epub Date: 2023-07-20DOI: 10.1007/s11336-023-09924-7
Matthias Kloft, Raphael Hartmann, Andreas Voss, Daniel W Heck
{"title":"The Dirichlet Dual Response Model: An Item Response Model for Continuous Bounded Interval Responses.","authors":"Matthias Kloft, Raphael Hartmann, Andreas Voss, Daniel W Heck","doi":"10.1007/s11336-023-09924-7","DOIUrl":"10.1007/s11336-023-09924-7","url":null,"abstract":"<p><p>Standard response formats such as rating or visual analogue scales require respondents to condense distributions of latent states or behaviors into a single value. Whereas this is suitable to measure central tendency, it neglects the variance of distributions. As a remedy, variability may be measured using interval-response formats, more specifically the dual-range slider (RS2). Given the lack of an appropriate item response model for the RS2, we develop the Dirichlet dual response model (DDRM), an extension of the beta response model (BRM; Noel & Dauvier in Appl Psychol Meas, 31:47-73, 2007). We evaluate the DDRM's performance by assessing parameter recovery in a simulation study. Results indicate overall good parameter recovery, although parameters concerning interval width (which reflect variability in behavior or states) perform worse than parameters concerning central tendency. We also test the model empirically by jointly fitting the BRM and the DDRM to single-range slider (RS1) and RS2 responses for two Extraversion scales. While the DDRM has an acceptable fit, it shows some misfit regarding the RS2 interval widths. Nonetheless, the model indicates substantial differences between respondents concerning variability in behavior. High correlations between person parameters of the BRM and DDRM suggest convergent validity between the RS1 and the RS2 interval location. Both the simulation and the empirical study demonstrate that the latent parameter space of the DDRM addresses an important issue of the RS2 response format, namely, the scale-inherent interdependence of interval location and interval width (i.e., intervals at the boundaries are necessarily smaller).</p>","PeriodicalId":54534,"journal":{"name":"Psychometrika","volume":"88 3","pages":"888-916"},"PeriodicalIF":2.9,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10444675/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10628954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PsychometrikaPub Date : 2023-09-01Epub Date: 2023-06-14DOI: 10.1007/s11336-023-09922-9
David J Hessen
{"title":"Fitting and Testing Log-Linear Subpopulation Models with Known Support.","authors":"David J Hessen","doi":"10.1007/s11336-023-09922-9","DOIUrl":"10.1007/s11336-023-09922-9","url":null,"abstract":"<p><p>In this paper, the support of the joint probability distribution of categorical variables in the total population is treated as unknown. From a general total population model with unknown support, a general subpopulation model with its support equal to the set of all observed score patterns is derived. In maximum likelihood estimation of the parameters of any such subpopulation model, the evaluation of the log-likelihood function only requires the summation over a number of terms equal to at most the sample size. It is made clear that the parameters of a hypothesized total population model are consistently and asymptotically efficiently estimated by the values that maximize the log-likelihood function of the corresponding subpopulation model. Next, new likelihood ratio goodness-of-fit tests are proposed as alternatives to the Pearson chi-square goodness-of-fit test and the likelihood ratio test against the saturated model. In a simulation study, the asymptotic bias and efficiency of maximum likelihood estimators and the asymptotic performance of the goodness-of-fit tests are investigated.</p>","PeriodicalId":54534,"journal":{"name":"Psychometrika","volume":"88 3","pages":"917-939"},"PeriodicalIF":2.9,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10444670/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10627476","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}