{"title":"Examining the Psychometric Impact of Targeted and Random Double-Scoring in Mixed-Format Assessments","authors":"Yangmeng Xu, Stefanie A. Wind","doi":"10.1111/emip.12636","DOIUrl":"https://doi.org/10.1111/emip.12636","url":null,"abstract":"<p>Double-scoring constructed-response items is a common but costly practice in mixed-format assessments. This study explored the impacts of Targeted Double-Scoring (TDS) and random double-scoring procedures on the quality of psychometric outcomes, including student achievement estimates, person fit, and student classifications under various conditions that reflect operational performance assessments. Using a simulation study, our results suggest no notable advantages for TDS over the random double-scoring approach across various psychometric outcomes, regardless of conditions related to student misfit, rater misfit, and rater severity. This study holds significant implications for mixed-format assessments, offering insights into a comprehensive evaluation of double-scoring methods. We recommend that researchers consider these findings when considering among double-scoring procedures.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"44 1","pages":"18-30"},"PeriodicalIF":2.7,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143424018","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Commentary: What Is the Breadth of “Educational Measurement?”","authors":"Marjorie Wine, Alexander M. Hoffman","doi":"10.1111/emip.12627","DOIUrl":"https://doi.org/10.1111/emip.12627","url":null,"abstract":"<p>The work of educational measurement is a highly collaborative endeavor that brings together professionals from many disciplines. While the introduction of the “Foundational Competencies in Educational Measurement” acknowledges this, the explanation of the framework itself falls short in acknowledging the competencies and skills of those from disciplines other than psychometrics, such as content development professionals (CDPs). Therefore, it is unable to sufficiently address the nature of validation work or other work not led by psychometricians. It also underexplores the vital competencies that underlie effective collaboration. As a result, it defines the competencies of psychometric work instead of the larger field of educational measurement.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"43 3","pages":"23-26"},"PeriodicalIF":2.7,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142404726","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Eduardo J. Crespo Cruz, Aria Immanuel, Lisa A. Keller, Ketan, Kimberly McIntee, Fernando José Mena Serrano, Stephen G. Sireci, Nate Smith, Javier Suárez-Álvarez, Craig S. Wells, Rebecca Woodland, April L. Zenisky
{"title":"Commentary: What Is Truly Foundational?","authors":"Eduardo J. Crespo Cruz, Aria Immanuel, Lisa A. Keller, Ketan, Kimberly McIntee, Fernando José Mena Serrano, Stephen G. Sireci, Nate Smith, Javier Suárez-Álvarez, Craig S. Wells, Rebecca Woodland, April L. Zenisky","doi":"10.1111/emip.12633","DOIUrl":"https://doi.org/10.1111/emip.12633","url":null,"abstract":"<p>The Task Force on Foundational Competencies in Educational Measurement has produced a set of foundational competencies and invited comment on the document. The students and faculty at the University of Massachusetts Amherst provide their comments and critique of the proposed competencies. Both students and faculty agree that there needs to be more specificity regarding the purpose of the document, the nature of the data used to produce the document, and the definition of the relevant terms. Additionally, attention should be paid to the international context, and the role of artificial intelligence and machine learning. The authors acknowledge the contribution of the draft of the foundational competencies and look forward to more conversation regarding this topic.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"43 3","pages":"52-55"},"PeriodicalIF":2.7,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142404651","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Past, Present, and Future of Large-Scale Assessment Consortia","authors":"Derek C. Briggs","doi":"10.1111/emip.12634","DOIUrl":"10.1111/emip.12634","url":null,"abstract":"<p>This article provides a history of the two large-scale assessment consortia that were funded in 2010 as part of the Race to the Top Competition, the Partnership for the Assessment of Readiness for College and Career (PARCC), and the Smarter-Balanced Assessment Consortium (SBAC). I compare the goals the consortia were funded to meet between 2011 and 2015 with the assessment systems PARCC and SBAC were able to deliver. Two notable accomplishments of both consortia were the development of high-quality summative assessments that were carefully designed to be accessible to all test-takers. I describe factors that led many states to withdraw from consortia membership between 2011 and 2015, and which led to the effective dissolution of PARCC. I discuss lessons learned from serving on the PARCC and SBAC technical advisory committees, and make a case for the value proposition of assessment consortia in the future.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"43 4","pages":"62-72"},"PeriodicalIF":2.7,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142211655","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Commentary: Where Does Classroom Assessment Fit in Educational Measurement?","authors":"Susan M. Brookhart","doi":"10.1111/emip.12626","DOIUrl":"10.1111/emip.12626","url":null,"abstract":"<p>This article is a commentary on the Foundational Competencies in Educational Measurement (Ackerman et al., 2024, hereafter the FCs) through the lens of classroom assessment. I focus on the place of classroom assessment in the FCs and in NCME and educational measurement more broadly.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"43 3","pages":"18-22"},"PeriodicalIF":2.7,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142211654","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Commentary: A Data-Driven Analysis of Recent Job Posts to Evaluate the Foundational Competencies","authors":"Ying Cheng, Bo Pei, Audrey Filonczuk, Anh Thu Le","doi":"10.1111/emip.12630","DOIUrl":"10.1111/emip.12630","url":null,"abstract":"<p>The Focal Article (Ackerman et al., 2024) provides a comprehensive framework and detailed guidelines to faculty, students, and measurement professionals. We used a data-driven approach that examines recent job posts (including full-time positions at academic institutions, nonprofit organizations, and for-profit companies, as well as internship positions) to evaluate the match between the skills required by the jobs and the Foundational Competencies and provide summaries and recommendations thereof.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"43 3","pages":"39-44"},"PeriodicalIF":2.7,"publicationDate":"2024-08-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142211656","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kamal Middlebrook, Laura S. Hamilton, Michael E. Walker
{"title":"Commentary: How Research and Testing Companies can Support Early-Career Measurement Professionals","authors":"Kamal Middlebrook, Laura S. Hamilton, Michael E. Walker","doi":"10.1111/emip.12631","DOIUrl":"10.1111/emip.12631","url":null,"abstract":"","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"43 3","pages":"45-48"},"PeriodicalIF":2.7,"publicationDate":"2024-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142211658","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Commentary: Modernizing Educational Assessment Training for Changing Job Markets","authors":"André A. Rupp","doi":"10.1111/emip.12629","DOIUrl":"10.1111/emip.12629","url":null,"abstract":"","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"43 3","pages":"33-38"},"PeriodicalIF":2.7,"publicationDate":"2024-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142211659","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}