{"title":"在混合形式评估中有针对性和随机双重评分对心理测量学的影响","authors":"Yangmeng Xu, Stefanie A. Wind","doi":"10.1111/emip.12636","DOIUrl":null,"url":null,"abstract":"<p>Double-scoring constructed-response items is a common but costly practice in mixed-format assessments. This study explored the impacts of Targeted Double-Scoring (TDS) and random double-scoring procedures on the quality of psychometric outcomes, including student achievement estimates, person fit, and student classifications under various conditions that reflect operational performance assessments. Using a simulation study, our results suggest no notable advantages for TDS over the random double-scoring approach across various psychometric outcomes, regardless of conditions related to student misfit, rater misfit, and rater severity. This study holds significant implications for mixed-format assessments, offering insights into a comprehensive evaluation of double-scoring methods. We recommend that researchers consider these findings when considering among double-scoring procedures.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"44 1","pages":"18-30"},"PeriodicalIF":2.7000,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Examining the Psychometric Impact of Targeted and Random Double-Scoring in Mixed-Format Assessments\",\"authors\":\"Yangmeng Xu, Stefanie A. Wind\",\"doi\":\"10.1111/emip.12636\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Double-scoring constructed-response items is a common but costly practice in mixed-format assessments. This study explored the impacts of Targeted Double-Scoring (TDS) and random double-scoring procedures on the quality of psychometric outcomes, including student achievement estimates, person fit, and student classifications under various conditions that reflect operational performance assessments. Using a simulation study, our results suggest no notable advantages for TDS over the random double-scoring approach across various psychometric outcomes, regardless of conditions related to student misfit, rater misfit, and rater severity. This study holds significant implications for mixed-format assessments, offering insights into a comprehensive evaluation of double-scoring methods. We recommend that researchers consider these findings when considering among double-scoring procedures.</p>\",\"PeriodicalId\":47345,\"journal\":{\"name\":\"Educational Measurement-Issues and Practice\",\"volume\":\"44 1\",\"pages\":\"18-30\"},\"PeriodicalIF\":2.7000,\"publicationDate\":\"2024-10-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Educational Measurement-Issues and Practice\",\"FirstCategoryId\":\"95\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1111/emip.12636\",\"RegionNum\":4,\"RegionCategory\":\"教育学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION & EDUCATIONAL RESEARCH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Educational Measurement-Issues and Practice","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/emip.12636","RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
Examining the Psychometric Impact of Targeted and Random Double-Scoring in Mixed-Format Assessments
Double-scoring constructed-response items is a common but costly practice in mixed-format assessments. This study explored the impacts of Targeted Double-Scoring (TDS) and random double-scoring procedures on the quality of psychometric outcomes, including student achievement estimates, person fit, and student classifications under various conditions that reflect operational performance assessments. Using a simulation study, our results suggest no notable advantages for TDS over the random double-scoring approach across various psychometric outcomes, regardless of conditions related to student misfit, rater misfit, and rater severity. This study holds significant implications for mixed-format assessments, offering insights into a comprehensive evaluation of double-scoring methods. We recommend that researchers consider these findings when considering among double-scoring procedures.