{"title":"Assessment of fit of item response theory models: A critical review of the status quo and some future directions.","authors":"Sandip Sinharay, Scott Monroe","doi":"10.1111/bmsp.12378","DOIUrl":"https://doi.org/10.1111/bmsp.12378","url":null,"abstract":"<p><p>This paper provides a literature review of assessment of fit of item response theory models. Various types of fit procedures for item response theory models are reviewed, with a focus on their advantages and disadvantages. Real data examples are used to demonstrate some of the fit procedures. Recommendations are provided for researchers and practitioners who are interested in assessing the fit of item response theory models.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":" ","pages":""},"PeriodicalIF":1.5,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142933352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The generalized Hausman test for detecting non-normality in the latent variable distribution of the two-parameter IRT model.","authors":"Lucia Guastadisegni, Silvia Cagnone, Irini Moustaki, Vassilis Vasdekis","doi":"10.1111/bmsp.12379","DOIUrl":"https://doi.org/10.1111/bmsp.12379","url":null,"abstract":"<p><p>This paper introduces the generalized Hausman test as a novel method for detecting the non-normality of the latent variable distribution of the unidimensional latent trait model for binary data. The test utilizes the pairwise maximum likelihood estimator for the parameters of the latent trait model, which assumes normality of the latent variable, and the maximum likelihood estimator obtained under a semi-non-parametric framework, allowing for a more flexible distribution of the latent variable. The performance of the generalized Hausman test is evaluated through a simulation study and compared with other test statistics available in the literature for testing latent variable distribution fit and an overall goodness-of-fit test statistic. Additionally, three information criteria are used to select the best-fitted model. The simulation results show that the generalized Hausman test outperforms the other tests under most conditions. However, the results obtained from the information criteria are somewhat contradictory under certain conditions, suggesting a need for further investigation and interpretation. The proposed test statistics are used in three datasets.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":" ","pages":""},"PeriodicalIF":1.5,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142900633","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Siqi He, Steven Andrew Culpepper, Jeffrey A Douglas
{"title":"A sparse latent class model incorporating response times.","authors":"Siqi He, Steven Andrew Culpepper, Jeffrey A Douglas","doi":"10.1111/bmsp.12380","DOIUrl":"https://doi.org/10.1111/bmsp.12380","url":null,"abstract":"<p><p>Diagnostic models (DM) have been widely used to classify respondents' latent attributes in cognitive and non-cognitive assessments. The integration of response times (RTs) with DM presents additional evidence to understand respondents' problem-solving behaviours. While recent research has explored using sparse latent class models (SLCM) to infer the latent structure of items based on item responses, the incorporation of RT data within these models remains underexplored. This study extends the SLCM framework to include RT, relaxing the conditional independence assumption between RT and latent attributes given individual speed. This adaptation provides a more flexible framework for jointly modelling RT and item responses. While the proposed model holds promise for applications in educational assessment, this study applied the model to the Fisher Temperament Inventory, yielding findings that provide a novel perspective on utilizing DM with RT in personality assessments. Additionally, a Gibbs sampling algorithm is proposed for parameter estimation. Results from Monte Carlo simulations demonstrate the algorithm's accuracy and efficiency.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":" ","pages":""},"PeriodicalIF":1.5,"publicationDate":"2024-12-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142900610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Eduardo Alarcón-Bustamante, Jorge González, David Torres Irribarra, Ernesto San Martín
{"title":"From missing data to informative GPA predictions: Navigating selection process beliefs with the partial identifiability approach","authors":"Eduardo Alarcón-Bustamante, Jorge González, David Torres Irribarra, Ernesto San Martín","doi":"10.1111/bmsp.12377","DOIUrl":"10.1111/bmsp.12377","url":null,"abstract":"<p>The extent to which college admissions test scores can forecast college grade point average (GPA) is often evaluated in predictive validity studies using regression analyses. A problem in college admissions processes is that we observe test scores for all the applicants; however, we cannot observe the GPA of applicants who were not selected. The standard solution to tackle this problem has relied upon strong assumptions to identify the exact value of the regression function in the presence of missing data. In this paper, we present an alternative approach based on the theory of partial identifiability that considers a variety of milder assumptions to learn about the regression function. Using a university admissions dataset we illustrate how results can vary as a function of the assumptions that one is willing to make about the selection process.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":"78 2","pages":"647-671"},"PeriodicalIF":1.5,"publicationDate":"2024-12-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142883680","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yi-An Zhu, Jingwan Xu, Daxun Wang, Xin Li, Yan Cai, Dongbo Tu
{"title":"A ranking forced choice diagnostic classification model for psychological assessment using forced choice questionnaires","authors":"Yi-An Zhu, Jingwan Xu, Daxun Wang, Xin Li, Yan Cai, Dongbo Tu","doi":"10.1111/bmsp.12376","DOIUrl":"10.1111/bmsp.12376","url":null,"abstract":"<p>The diagnostic classification model (DCM) has been widely utilized in non-cognitive tests, offering diagnostic information on latent attributes. However, the model's reliance on single-stimulus (SS) items may lead to response biases (e.g., social desirability), jeopardizing the psychometric properties. As an alternative to SS scales, forced choice questionnaires (FCQ) can effectively control response biases. The combination of FCQs and the DCM not only circumvents response bias but also yields fine-grained diagnostic information on latent attributes. To the best of our knowledge, only one study (Huang, <i>Educ. Psychol. Meas.</i>, <i>83</i>, 2022, 146) has explored this topic and developed a DCM for forced choice (FC) items. However, the existing model has limitations in terms of its modelling assumption, the FC format and the number of attributes measured by statement. To address these limitations, this study proposes a ranking FC-DCM that (1) adopts a generalized assumption, (2) covers all FC formats and (3) eases the limitation on the number of attributes measured by each statement. The simulation study demonstrated that the proposed model exhibited satisfactory person and item parameter recovery under all conditions. This study provided an illustrative example by developing an FC version questionnaire to further explore the applications and advantages of the proposed model in real-world settings.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":"78 2","pages":"617-646"},"PeriodicalIF":1.5,"publicationDate":"2024-12-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142830267","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Frequency-adjusted borders ordinal forest: A novel tree ensemble method for ordinal prediction","authors":"Philip Buczak","doi":"10.1111/bmsp.12375","DOIUrl":"10.1111/bmsp.12375","url":null,"abstract":"<p>Ordinal responses commonly occur in psychology, e.g., through school grades or rating scales. Where traditionally parametric statistical models like the proportional odds model have been used, machine learning (ML) methods such as random forest (RF) are increasingly employed for ordinal prediction. With new developments in assessment and new data sources yielding increasing quantities of data in the psychological sciences, such ML approaches promise high predictive performance. As RF does not inherently account for ordinality, several extensions have been proposed. A promising approach lies in assigning optimized numeric scores to the ordinal response categories and using regression RF. However, these optimization procedures are computationally expensive and have been shown to yield only situational benefit. In this work, I propose Frequency-Adjusted Borders Ordinal Forest (fabOF), a novel tree ensemble method for ordinal prediction forgoing extensive optimization while offering improved predictive performance in simulation and an illustrative example of student performance. To aid interpretation, I additionally introduce a permutation variable importance measure for fabOF tailored towards ordinal prediction. When applied to the illustrative example, an interest in higher education, mother's education, and study time are identified as important predictors of student performance. The presented methodology is made available through an accompanying <span>R</span> package.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":"78 2","pages":"594-616"},"PeriodicalIF":1.5,"publicationDate":"2024-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bmsp.12375","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142796610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Leonie V D E Vogelsmeier, Irina Uglanova, Manuel T Rein, Esther Ulitzsch
{"title":"Investigating dynamics in attentive and inattentive responding together with their contextual correlates using a novel mixture IRT model for intensive longitudinal data.","authors":"Leonie V D E Vogelsmeier, Irina Uglanova, Manuel T Rein, Esther Ulitzsch","doi":"10.1111/bmsp.12373","DOIUrl":"https://doi.org/10.1111/bmsp.12373","url":null,"abstract":"<p><p>In ecological momentary assessment (EMA), respondents answer brief questionnaires about their current behaviours or experiences several times per day across multiple days. The frequent measurement enables a thorough grasp of the dynamics inherent in psychological constructs, but it also increases respondent burden. To lower this burden, respondents may engage in careless and insufficient effort responding (C/IER), leaving data contaminated with responses that do not reflect what researchers want to measure. We introduce a novel approach to investigating C/IER in EMA data. Our approach combines a confirmatory mixture item response theory model separating C/IER from attentive behaviour with latent Markov factor analysis. This enables gauging the occurrence of C/IER and studying transitions among states of different response behaviours including their contextual correlates. The approach can be implemented using R packages. An empirical application showcases the approach's efficacy in pinpointing C/IER instances and gaining insights into their underlying causes. We showcase that the approach identifies various C/IER response patterns but requires heterogeneous and negatively worded items to detect straightlining. In a simulation investigating robustness against unaccounted for changes in measurement models underlying attentive responses, the approach proved robust against heterogeneity in loading patterns but not against heterogeneity in factor structures. Extensions to accommodate the latter are discussed.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":" ","pages":""},"PeriodicalIF":1.5,"publicationDate":"2024-12-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142792957","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Statistical foundations of person parameter estimation in the Thurstonian IRT model for forced-choice and pairwise comparison designs","authors":"Safir Yousfi","doi":"10.1111/bmsp.12364","DOIUrl":"10.1111/bmsp.12364","url":null,"abstract":"<p>The statistical foundations of person parameter estimation for the multivariate Thurstonian item response theory (TIRT) model of pairwise comparison and forced-choice (FC) ranking data are elaborated, and several misconceptions in IRT and TIRT are addressed. It is shown that directional information (i.e. multivariate information as defined by Reckase & Kinley, 1991; <i>Applied Psychological Measurement</i>, 15, 361) is not suited to quantify the precision of the estimates unless the Fisher information matrix is diagonal. The asymptotic covariance can be quantified by the inverse Fisher information matrix if the genuine likelihood is used and by the inverse Godambe information for independence likelihood estimation that results from ignoring within-block dependencies of pairwise comparisons. Analytical expressions are provided for the genuine likelihood and the Fisher information matrix for a generalized TIRT model that comprises binary pairwise comparison and ranking designs, which enables maximum likelihood estimation (MLE) and Bayesian estimation (maximum a posteriori probability with normal and Jeffreys prior) of person parameters. The bias of the MLE is quantified, and methods of bias prevention and bias correction are introduced. The correct marginal likelihood of graded pairwise comparisons is provided that might be used for person parameter estimation based on the independence likelihood.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":"78 2","pages":"555-593"},"PeriodicalIF":1.5,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142734861","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A new Q-matrix validation method based on signal detection theory","authors":"Jia Li, Ping Chen","doi":"10.1111/bmsp.12371","DOIUrl":"10.1111/bmsp.12371","url":null,"abstract":"<p>The Q-matrix is a crucial component of cognitive diagnostic theory and an important basis for the research and practical application of cognitive diagnosis. In practice, the Q-matrix is typically developed by domain experts and may contain some misspecifications, so it needs to be refined using Q-matrix validation methods. Based on signal detection theory, this paper puts forward a new Q-matrix validation method (i.e., <span></span><math>\u0000 <semantics>\u0000 <mrow>\u0000 <mi>β</mi>\u0000 </mrow>\u0000 </semantics></math> method) and then conducts a simulation study to compare the new method with existing methods. The results show that when the model is DINA (deterministic inputs, noisy ‘and’ gate), the <span></span><math>\u0000 <semantics>\u0000 <mrow>\u0000 <mi>β</mi>\u0000 </mrow>\u0000 </semantics></math> method outperforms the existing methods under all conditions; under the generalized DINA (G-DINA) model, the method still has the highest validation rate when the sample size is small, and the item quality is high or the rate of Q-matrix misspecification is ≥.4. Finally, a sub-dataset of the PISA 2000 reading assessment is analysed to evaluate the reliability of the <span></span><math>\u0000 <semantics>\u0000 <mrow>\u0000 <mi>β</mi>\u0000 </mrow>\u0000 </semantics></math> method.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":"78 2","pages":"522-554"},"PeriodicalIF":1.5,"publicationDate":"2024-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142677821","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}