Applied Measurement in Education最新文献_第4页

Maintaining Score Scales Over Time: A Comparison of Five Scoring Methods 随着时间的推移保持分数尺度:五种评分方法的比较

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2023-01-02 DOI: 10.1080/08957347.2023.2172015

S. Y. Kim, Won‐Chan Lee

引用次数: 0

Accuracy and Sensitivity of Coefficient Alpha and Its Alternatives with Unidimensional and Contaminated Scales 一维和污染尺度下系数Alpha及其替代方案的精度和灵敏度

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2023-01-02 DOI: 10.1080/08957347.2023.2172016

Leifeng Xiao, K. Hau

引用次数: 1

Using Bayesian Networks for Cognitive Assessment of Student Understanding of Buoyancy: A Granular Hierarchy Model 使用贝叶斯网络对学生浮力理解的认知评估：一个粒度层次模型

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2023-01-02 DOI: 10.1080/08957347.2023.2172014

L. Wang, Sun Xiao Jian, Yan Lou Liu, Tao Xin

引用次数: 0

Are Large Admissions Test Coaching Effects Widespread? A Longitudinal Analysis of Admissions Test Scores 大型招生考试辅导效果广泛吗？招生考试成绩的纵向分析

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2023-01-02 DOI: 10.1080/08957347.2023.2172018

Jeffrey A. Dahlke, P. Sackett, N. Kuncel

引用次数: 0

Dissecting knowledge, guessing, and blunder in multiple choice assessments. 剖析多项选择评估中的知识、猜测和失误。

IF 1.1 4区教育学

Applied Measurement in Education Pub Date : 2023-01-01 Epub Date: 2023-02-21 DOI: 10.1080/08957347.2023.2172017

Rashid M Abu-Ghazalah, David N Dubins, Gregory M K Poon

{"title":"Dissecting knowledge, guessing, and blunder in multiple choice assessments.","authors":"Rashid M Abu-Ghazalah, David N Dubins, Gregory M K Poon","doi":"10.1080/08957347.2023.2172017","DOIUrl":"10.1080/08957347.2023.2172017","url":null,"abstract":"Multiple choice results are inherently probabilistic outcomes, as correct responses reflect a combination of knowledge and guessing, while incorrect responses additionally reflect blunder, a confidently committed mistake. To objectively resolve knowledge from responses in an MC test structure, we evaluated probabilistic models that explicitly account for guessing, knowledge and blunder using eight assessments (>9,000 responses) from an undergraduate biotechnology curriculum. A Bayesian implementation of the models, aimed at assessing their robustness to prior beliefs in examinee knowledge, showed that explicit estimators of knowledge are markedly sensitive to prior beliefs with scores as sole input. To overcome this limitation, we examined self-ranked confidence as a proxy knowledge indicator. For our test set, three levels of confidence resolved test performance. Responses rated as least confident were correct more frequently than expected from random selection, reflecting partial knowledge, but were balanced by blunder among the most confident responses. By translating evidence-based guessing and blunder rates to pass marks that statistically qualify a desired level of examinee knowledge, our approach finds practical utility in test analysis and design.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"36 1","pages":"80-98"},"PeriodicalIF":1.1,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10201919/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9522330","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Personality Aspects and the Underprediction of Women’s Academic Performance 人格因素与女性学习成绩预测不足

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2022-10-02 DOI: 10.1080/08957347.2022.2155652

You Zhou, P. Sackett, Thomas Brothen

引用次数: 0

An Examination of Individual Ability Estimation and Classification Accuracy Under Rapid Guessing Misidentifications 快速猜错识别下的个人能力估计与分类准确度检验

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2022-10-02 DOI: 10.1080/08957347.2022.2155653

Joseph A. Rios

{"title":"An Examination of Individual Ability Estimation and Classification Accuracy Under Rapid Guessing Misidentifications","authors":"Joseph A. Rios","doi":"10.1080/08957347.2022.2155653","DOIUrl":"https://doi.org/10.1080/08957347.2022.2155653","url":null,"abstract":"ABSTRACT To mitigate the deleterious effects of rapid guessing (RG) on ability estimates, several rescoring procedures have been proposed. Underlying many of these procedures is the assumption that RG is accurately identified. At present, there have been minimal investigations examining the utility of rescoring approaches when RG is misclassified, and individual scores are reported. To address this limitation, the present simulation study investigates the effect of RG misclassifications on individual examinee ability estimate bias and classification accuracy when using effort-moderated (EM) scoring. This objective is accomplished by manipulating simulee ability level, RG rate, as well as misclassification type and percentage. Results showed that EM scoring significantly improved ability inferences for examinees engaging in RG; however, the effectiveness of this approach was largely dependent on misclassification type. Specifically, across ability levels, bias tended to be on average lower when falsely classifying effortful responses as RG. Although EM scoring improved bias, it was susceptible to elevated false-positive classifications of ability under high RG.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"35 1","pages":"300 - 312"},"PeriodicalIF":1.5,"publicationDate":"2022-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42107151","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Comparison of Methods for Identifying Differential Step Functioning with Polytomous Item Response Data 基于多同构项目响应数据的差分阶跃函数识别方法比较

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2022-10-02 DOI: 10.1080/08957347.2022.2155650

Holmes W. Finch

{"title":"Comparison of Methods for Identifying Differential Step Functioning with Polytomous Item Response Data","authors":"Holmes W. Finch","doi":"10.1080/08957347.2022.2155650","DOIUrl":"https://doi.org/10.1080/08957347.2022.2155650","url":null,"abstract":"ABSTRACT Much research has been devoted to identification of differential item functioning (DIF), which occurs when the item responses for individuals from two groups differ after they are conditioned on the latent trait being measured by the scale. There has been less work examining differential step functioning (DSF), which is present for polytomous items when the conditional likelihood of responses to specific categories differ between groups. DSF impacts estimation of the measured trait and reduces the effectiveness of standard DIF detection methods. The purpose of this simulation study was to extend upon earlier work by comparing several methods for detecting the presence of DSF in polytomous items, including an approach based on the lasso estimation of the generalized partial credit model. Results show that the lasso GPCM technique controlled the Type I error rate while yielding power rates somewhat lower than logistic regression and the MIMIC model, which were not able to control the Type I error rate in some conditions. An empirical example is also presented, and implications of this study for practice are discussed.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"35 1","pages":"255 - 271"},"PeriodicalIF":1.5,"publicationDate":"2022-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47299711","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Performance Decline as an Indicator of Generalized Test-Taking Disengagement 性能下降作为广义测试脱离的一个指标

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2022-10-02 DOI: 10.1080/08957347.2022.2155651

S. Wise, G. Kingsbury

引用次数: 3

When Should Individual Ability Estimates Be Reported if Rapid Guessing Is Present? 如果存在快速猜测，应该在什么时候报告个人能力评估?

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2022-07-26 DOI: 10.1080/08957347.2022.2103138

Joseph A. Rios

{"title":"When Should Individual Ability Estimates Be Reported if Rapid Guessing Is Present?","authors":"Joseph A. Rios","doi":"10.1080/08957347.2022.2103138","DOIUrl":"https://doi.org/10.1080/08957347.2022.2103138","url":null,"abstract":"ABSTRACTTesting programs are confronted with the decision of whether to report individual scores for examinees that have engaged in rapid guessing (RG). As noted by the Standards for Educational and Psychological Testing, this decision should be based on a documented criterion that determines score exclusion. To this end, a number of heuristic criteria (e.g., exclude all examinees with RG rates of 10%) have been adopted in the literature. Given that these criteria lack strong methodological support, the objective of this simulation study was to evaluate their appropriateness in terms of individual ability estimate and classification accuracy when manipulating both assessment and RG characteristics. The findings provide evidence that employing a common criterion for all examinees may be an ineffective strategy because a given RG percentage may have differing degrees of biasing effects based on test difficulty, examinee ability, and RG pattern. These results suggest that practitioners may benefit from establishing context-specific exclusion criteria that consider test purpose, score use, and targeted examinee trait levels.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"65 11","pages":""},"PeriodicalIF":1.5,"publicationDate":"2022-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138495008","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1