{"title":"Exploring examinees' responses to constructed response items with a supervised topic model","authors":"Seohyun Kim, Zhenqiu Lu, Allan S. Cohen","doi":"10.1111/bmsp.12319","DOIUrl":"10.1111/bmsp.12319","url":null,"abstract":"<p>Textual data are increasingly common in test data as many assessments include constructed response (CR) items as indicators of participants' understanding. The development of techniques based on natural language processing has made it possible for researchers to rapidly analyse large sets of textual data. One family of statistical techniques for this purpose are probabilistic topic models. Topic modelling is a technique for detecting the latent topic structure in a collection of documents and has been widely used to analyse texts in a variety of areas. The detected topics can reveal primary themes in the documents, and the relative use of topics can be useful in investigating the variability of the documents. Supervised latent Dirichlet allocation (SLDA) is a popular topic model in that family that jointly models textual data and paired responses such as could occur with participants' textual answers to CR items and their rubric-based scores. SLDA has an assumption of a homogeneous relationship between textual data and paired responses across all documents. This approach, while useful for some purposes, may not be satisfied for situations in which a population has subgroups that have different relationships. In this study, we introduce a new supervised topic model that incorporates finite-mixture modelling into the SLDA. This new model can detect latent groups of participants that have different relationships between their textual responses and associated scores. The model is illustrated with an example from an analysis of a set of textual responses and paired scores from a middle grades assessment of science inquiry knowledge. A simulation study is presented to investigate the performance of the proposed model under practical testing conditions.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":null,"pages":null},"PeriodicalIF":2.6,"publicationDate":"2023-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bmsp.12319","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10225777","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lu Ou, Michael D. Hunter, Zhaohua Lu, Cynthia A. Stifter, Sy-Miin Chow
{"title":"Estimation of nonlinear mixed-effects continuous-time models using the continuous-discrete extended Kalman filter","authors":"Lu Ou, Michael D. Hunter, Zhaohua Lu, Cynthia A. Stifter, Sy-Miin Chow","doi":"10.1111/bmsp.12318","DOIUrl":"10.1111/bmsp.12318","url":null,"abstract":"<p>Many intensive longitudinal measurements are collected at irregularly spaced time intervals, and involve complex, possibly nonlinear and heterogeneous patterns of change. Effective modelling of such change processes requires continuous-time differential equation models that may be nonlinear and include mixed effects in the parameters. One approach of fitting such models is to define random effect variables as additional latent variables in a stochastic differential equation (SDE) model of choice, and use estimation algorithms designed for fitting SDE models, such as the continuous-discrete extended Kalman filter (CDEKF) approach implemented in the <i>dynr</i> R package, to estimate the random effect variables as latent variables. However, this approach's efficacy and identification constraints in handling mixed-effects SDE models have not been investigated. In the current study, we analytically inspect the identification constraints of using the CDEKF approach to fit nonlinear mixed-effects SDE models; extend a published model of emotions to a nonlinear mixed-effects SDE model as an example, and fit it to a set of irregularly spaced ecological momentary assessment data; and evaluate the feasibility of the proposed approach to fit the model through a Monte Carlo simulation study. Results show that the proposed approach produces reasonable parameter and standard error estimates when some identification constraint is met. We address the effects of sample size, process noise variance, and data spacing conditions on estimation results.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":null,"pages":null},"PeriodicalIF":2.6,"publicationDate":"2023-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10226412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using item scores and response times in person-fit assessment","authors":"Kylie Gorney, Sandip Sinharay, Xiang Liu","doi":"10.1111/bmsp.12320","DOIUrl":"10.1111/bmsp.12320","url":null,"abstract":"<p>The use of joint models for item scores and response times is becoming increasingly popular in educational and psychological testing. In this paper, we propose two new person-fit statistics for such models in order to detect aberrant behaviour. The first statistic is computed by combining two existing person-fit statistics: one for the item scores, and one for the item response times. The second statistic is computed directly using the likelihood function of the joint model. Using detailed simulations, we show that the empirical null distributions of the new statistics are very close to the theoretical null distributions, and that the new statistics tend to be more powerful than several existing statistics for item scores and/or response times. A real data example is also provided using data from a licensure examination.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":null,"pages":null},"PeriodicalIF":2.6,"publicationDate":"2023-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bmsp.12320","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10156809","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluating the performance of existing and novel equivalence tests for fit indices in structural equation modelling","authors":"Nataly Beribisky, Robert A. Cribbie","doi":"10.1111/bmsp.12317","DOIUrl":"10.1111/bmsp.12317","url":null,"abstract":"<p>It has been suggested that equivalence testing (otherwise known as negligible effect testing) should be used to evaluate model fit within structural equation modelling (SEM). In this study, we propose novel variations of equivalence tests based on the popular root mean squared error of approximation and comparative fit index fit indices. Using Monte Carlo simulations, we compare the performance of these novel tests to other existing equivalence testing-based fit indices in SEM, as well as to other methods commonly used to evaluate model fit. Results indicate that equivalence tests in SEM have good Type I error control and display considerable power for detecting well-fitting models in medium to large sample sizes. At small sample sizes, relative to traditional fit indices, equivalence tests limit the chance of supporting a poorly fitting model. We also present an illustrative example to demonstrate how equivalence tests may be incorporated in model fit reporting. Equivalence tests in SEM also have unique interpretational advantages compared to other methods of model fit evaluation. We recommend that equivalence tests be utilized in conjunction with descriptive fit indices to provide more evidence when evaluating model fit.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":null,"pages":null},"PeriodicalIF":2.6,"publicationDate":"2023-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bmsp.12317","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10134925","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"K-Plus anticlustering: An improved k-means criterion for maximizing between-group similarity","authors":"Martin Papenberg","doi":"10.1111/bmsp.12315","DOIUrl":"10.1111/bmsp.12315","url":null,"abstract":"<p>Anticlustering refers to the process of partitioning elements into disjoint groups with the goal of obtaining high between-group similarity and high within-group heterogeneity. Anticlustering thereby reverses the logic of its better known twin—cluster analysis—and is usually approached by maximizing instead of minimizing a clustering objective function. This paper presents <i>k</i>-plus, an extension of the classical <i>k</i>-means objective of maximizing between-group similarity in anticlustering applications. <i>K</i>-plus represents between-group similarity as discrepancy in distribution moments (means, variance, and higher-order moments), whereas the <i>k</i>-means criterion only reflects group differences with regard to means. While constituting a new criterion for anticlustering, it is shown that <i>k</i>-plus anticlustering can be implemented by optimizing the original <i>k</i>-means criterion after the input data have been augmented with additional variables. A computer simulation and practical examples show that <i>k</i>-plus anticlustering achieves high between-group similarity with regard to multiple objectives. In particular, optimizing between-group similarity with regard to variances usually does not compromise similarity with regard to means; the <i>k</i>-plus extension is therefore generally preferred over classical <i>k</i>-means anticlustering. Examples are given on how <i>k</i>-plus anticlustering can be applied to real norming data using the open source R package anticlust, which is freely available via CRAN.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":null,"pages":null},"PeriodicalIF":2.6,"publicationDate":"2023-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bmsp.12315","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9764395","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Enhancing measurement validity in diverse populations: Modern approaches to evaluating differential item functioning","authors":"Daniel J. Bauer","doi":"10.1111/bmsp.12316","DOIUrl":"10.1111/bmsp.12316","url":null,"abstract":"<p>When developing and evaluating psychometric measures, a key concern is to ensure that they accurately capture individual differences on the intended construct across the entire population of interest. Inaccurate assessments of individual differences can occur when responses to some items reflect not only the intended construct but also construct-irrelevant characteristics, like a person's race or sex. Unaccounted for, this <i>item bias</i> can lead to apparent differences on the scores that do not reflect true differences, invalidating comparisons between people with different backgrounds. Accordingly, empirically identifying which items manifest bias through the evaluation of <i>differential item functioning</i> (DIF) has been a longstanding focus of much psychometric research. The majority of this work has focused on evaluating DIF across two (or a few) groups. Modern conceptualizations of identity, however, emphasize its multi-determined and intersectional nature, with some aspects better represented as dimensional than categorical. Fortunately, many model-based approaches to modelling DIF now exist that allow for simultaneous evaluation of multiple background variables, including both continuous and categorical variables, and potential interactions among background variables. This paper provides a comparative, integrative review of these new approaches to modelling DIF and clarifies both the opportunities and challenges associated with their application in psychometric research.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":null,"pages":null},"PeriodicalIF":2.6,"publicationDate":"2023-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bmsp.12316","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10125818","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Assessment of generalised Bayesian structural equation models for continuous and binary data","authors":"Konstantinos Vamvourellis, Konstantinos Kalogeropoulos, Irini Moustaki","doi":"10.1111/bmsp.12314","DOIUrl":"10.1111/bmsp.12314","url":null,"abstract":"<p>The paper proposes a novel model assessment paradigm aiming to address shortcoming of posterior predictive <math>\u0000 <semantics>\u0000 <mrow>\u0000 <mi>p</mi>\u0000 </mrow>\u0000 </semantics></math>-values, which provide the default metric of fit for Bayesian structural equation modelling (BSEM). The model framework presented in the paper focuses on the approximate zero approach (<i>Psychological Methods</i>, <b>17</b>, 2012, 313), which involves formulating certain parameters (such as factor loadings) to be approximately zero through the use of informative priors, instead of explicitly setting them to zero. The introduced model assessment procedure monitors the out-of-sample predictive performance of the fitted model, and together with a list of guidelines we provide, one can investigate whether the hypothesised model is supported by the data. We incorporate scoring rules and cross-validation to supplement existing model assessment metrics for BSEM. The proposed tools can be applied to models for both continuous and binary data. The modelling of categorical and non-normally distributed continuous data is facilitated with the introduction of an item-individual random effect. We study the performance of the proposed methodology via simulation experiments as well as real data on the ‘Big-5’ personality scale and the Fagerstrom test for nicotine dependence.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":null,"pages":null},"PeriodicalIF":2.6,"publicationDate":"2023-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bmsp.12314","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9748065","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Testing indirect effect with a complete or incomplete dichotomous mediator","authors":"Fan Jia, Wei Wu, Po-Yi Chen","doi":"10.1111/bmsp.12313","DOIUrl":"10.1111/bmsp.12313","url":null,"abstract":"<p>Past methodological research on mediation analysis mainly focused on situations where all variables were complete and continuous. When issues of categorical data occur combined with missing data, more methodological considerations are involved. Specifically, appropriate decisions need to be made on estimation methods of the indirect effects and on confidence intervals for testing the indirect effects with accommodations of missing data. We compare strategies that address these issues based on a model with a dichotomous mediator, aiming to provide guidelines for researchers facing such challenges in practice.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":null,"pages":null},"PeriodicalIF":2.6,"publicationDate":"2023-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10063422","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Variational Bayes inference for hidden Markov diagnostic classification models","authors":"Kazuhiro Yamaguchi, Alfonso J. Martinez","doi":"10.1111/bmsp.12308","DOIUrl":"10.1111/bmsp.12308","url":null,"abstract":"<p>Diagnostic classification models (DCMs) can be used to track the cognitive learning states of students across multiple time points or over repeated measurements. This study developed an effective variational Bayes (VB) inference method for hidden Markov longitudinal general DCMs. The simulations performed in this study verified the validity of the proposed algorithm for satisfactorily recovering true parameters. Simulation and applied data analyses were conducted to compare the proposed VB method to Markov chain Monte Carlo (MCMC) sampling. The results revealed that the parameter estimates provided by the VB method were consistent with the MCMC method with the additional benefit of a faster estimation time. The comparative simulation also indicated differences between the two methods in terms of posterior standard deviation and coverage of 95% credible intervals. Thus, with limited computational resources and time, the proposed VB method can output estimations comparable to that of MCMC.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":null,"pages":null},"PeriodicalIF":2.6,"publicationDate":"2023-05-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9533717","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Auburn Jimenez, James Joseph Balamuta, Steven Andrew Culpepper
{"title":"A sequential exploratory diagnostic model using a Pólya-gamma data augmentation strategy","authors":"Auburn Jimenez, James Joseph Balamuta, Steven Andrew Culpepper","doi":"10.1111/bmsp.12307","DOIUrl":"10.1111/bmsp.12307","url":null,"abstract":"<p>Cognitive diagnostic models provide a framework for classifying individuals into latent proficiency classes, also known as attribute profiles. Recent research has examined the implementation of a Pólya-gamma data augmentation strategy binary response model using logistic item response functions within a Bayesian Gibbs sampling procedure. In this paper, we propose a sequential exploratory diagnostic model for ordinal response data using a logit-link parameterization at the category level and extend the Pólya-gamma data augmentation strategy to ordinal response processes. A Gibbs sampling procedure is presented for efficient Markov chain Monte Carlo (MCMC) estimation methods. We provide results from a Monte Carlo study for model performance and present an application of the model.</p>","PeriodicalId":55322,"journal":{"name":"British Journal of Mathematical & Statistical Psychology","volume":null,"pages":null},"PeriodicalIF":2.6,"publicationDate":"2023-05-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41170914","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}