{"title":"Generating Models for Item Preknowledge","authors":"Kylie Gorney, James A. Wollack","doi":"10.1111/jedm.12309","DOIUrl":"https://doi.org/10.1111/jedm.12309","url":null,"abstract":"","PeriodicalId":47871,"journal":{"name":"Journal of Educational Measurement","volume":" ","pages":""},"PeriodicalIF":1.3,"publicationDate":"2022-03-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46238645","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exploring the Impact of Random Guessing in Distractor Analysis","authors":"K. Jin, Wai‐Lok Siu, Xiaoting Huang","doi":"10.1111/jedm.12310","DOIUrl":"https://doi.org/10.1111/jedm.12310","url":null,"abstract":"","PeriodicalId":47871,"journal":{"name":"Journal of Educational Measurement","volume":" ","pages":""},"PeriodicalIF":1.3,"publicationDate":"2022-03-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45576317","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Editorial for JEM issue 58-4","authors":"","doi":"10.1111/jedm.12308","DOIUrl":"https://doi.org/10.1111/jedm.12308","url":null,"abstract":"","PeriodicalId":47871,"journal":{"name":"Journal of Educational Measurement","volume":"58 4","pages":"437"},"PeriodicalIF":1.3,"publicationDate":"2022-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"92369530","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Constructing a Robust Score Scale from IRT Scores with Informed Boundaries","authors":"Edison M. Choe, K. T. Han","doi":"10.1111/jedm.12307","DOIUrl":"https://doi.org/10.1111/jedm.12307","url":null,"abstract":"","PeriodicalId":47871,"journal":{"name":"Journal of Educational Measurement","volume":" ","pages":""},"PeriodicalIF":1.3,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44630096","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Explanatory Cognitive Diagnostic Modeling Incorporating Response Times","authors":"Xin Qiao, Hong Jiao","doi":"10.1111/jedm.12306","DOIUrl":"10.1111/jedm.12306","url":null,"abstract":"<p>This study proposes explanatory cognitive diagnostic model (CDM) jointly incorporating responses and response times (RTs) with the inclusion of item covariates related to both item responses and RTs. The joint modeling of item responses and RTs intends to provide more information for cognitive diagnosis while item covariates can be used to predict item parameters when item calibration is not feasible in diagnostic assessments or item parameter estimation errors could be too large due to small sample sizes for calibration. In addition, the inclusion of the item covariates allows the evaluation of cognitive theories underlying the test design in item development. Model parameter estimation is explored using the Bayesian Markov chain Monte Carlo (MCMC) method. A Monte Carlo simulation study is conducted to examine the parameter recovery of the proposed model under different simulated conditions in comparison to alternative competing models. Further, the application of the proposed model is illustrated using the Programme for International Student Assessment (PISA) 2012 problem-solving items modeling both item response and RT data. The study results indicate that model parameters can be well recovered using the MCMC algorithm and the explanatory CDM jointly incorporating item responses and RTs with item covariates holds promising applications in digital-based diagnostic assessments.</p>","PeriodicalId":47871,"journal":{"name":"Journal of Educational Measurement","volume":"58 4","pages":"564-585"},"PeriodicalIF":1.3,"publicationDate":"2022-02-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47546044","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Automated Test Assembly and Routing Rule for Multistage Adaptive Testing with Multidimensional Item Response Theory","authors":"Lingling Xu, Shiyu Wang, Yan Cai, Dongbo Tu","doi":"10.1111/jedm.12305","DOIUrl":"10.1111/jedm.12305","url":null,"abstract":"<p>Designing a multidimensional adaptive test (M-MST) based on a multidimensional item response theory (MIRT) model is critical to make full use of the advantages of both MST and MIRT in implementing multidimensional assessments. This study proposed two types of automated test assembly (ATA) algorithms and one set of routing rules that can facilitate the development of an M-MST. Different M-MST designs were developed based on the proposed ATA algorithms and routing rules and were evaluated through two sets of simulation studies. Study 1 used simulated item banks and considered a variety of testing factors, and results from which can inform us the theoretical performance of the proposed M-MST designs. Study 2 was designed based on a real multidimensional assessment. In addition, a MCAT was simulated as a baseline design to demonstrate the advantage of the proposed M-MST design in each study. Our simulation results indicate that the proposed ATA algorithms and routing rule can generate M-MSTs with a good-quality control and the same or even better ability estimation results than the MCAT given the same condition. This demonstrates the advantage of using M-MST for multidimensional assessment especially for the one that needs to satisfy many nonstatistical constraints.</p>","PeriodicalId":47871,"journal":{"name":"Journal of Educational Measurement","volume":"58 4","pages":"538-563"},"PeriodicalIF":1.3,"publicationDate":"2022-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48046621","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Fully Gibbs Sampling Algorithms for Bayesian Variable Selection in Latent Regression Models","authors":"K. Yamaguchi, Jihong Zhang","doi":"10.31234/osf.io/dfrxj","DOIUrl":"https://doi.org/10.31234/osf.io/dfrxj","url":null,"abstract":"This study proposed efficient Gibbs sampling algorithms for variable selection in a latent regression model under a unidimensional two-parameter logistic item response theory model. Three types of shrinkage priors were employed to obtain shrinkage estimates: double-exponential (i.e., Laplace), horseshoe, and horseshoe+ priors. These shrinkage priors were compared to a uniform prior case in both simulation and real data analysis. The simulation study revealed that two types of horseshoe priors had a smaller root mean square errors and shorter 95% credible interval lengths than double-exponential or uniform priors. In addition, the horseshoe prior+ was slightly more stable than the horseshoe prior. The real data example successfully proved the utility of horseshoe and horseshoe+ priors in selecting effective predictive covariates for math achievement. In the final section, we discuss the benefits and limitations of the three types of Bayesian variable selection methods.","PeriodicalId":47871,"journal":{"name":"Journal of Educational Measurement","volume":" ","pages":""},"PeriodicalIF":1.3,"publicationDate":"2021-12-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42439354","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Victoria Yaneva, Brian E. Clauser, Amy Morales, Miguel Paniagua
{"title":"Using Eye-Tracking Data as Part of the Validity Argument for Multiple-Choice Questions: A Demonstration","authors":"Victoria Yaneva, Brian E. Clauser, Amy Morales, Miguel Paniagua","doi":"10.1111/jedm.12304","DOIUrl":"10.1111/jedm.12304","url":null,"abstract":"<p>Eye-tracking technology can create a record of the location and duration of visual fixations as a test-taker reads test questions. Although the cognitive process the test-taker is using cannot be directly observed, eye-tracking data can support inferences about these unobserved cognitive processes. This type of information has the potential to support improved test design and to contribute to an overall validity argument for the inferences and uses made based on test scores. Although several authors have referred to the potential usefulness of eye-tracking data, there are relatively few published studies that provide examples of that use. In this paper, we report the results an eye-tracking study designed to evaluate how the presence of the options in multiple-choice questions impacts the way medical students responded to questions designed to evaluate clinical reasoning. Examples of the types of data that can be extracted are presented. We then discuss the implications of these results for evaluating the validity of inferences made based on the type of items used in this study.</p>","PeriodicalId":47871,"journal":{"name":"Journal of Educational Measurement","volume":"58 4","pages":"515-537"},"PeriodicalIF":1.3,"publicationDate":"2021-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/jedm.12304","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43557062","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}