Joe Betts, Cary Lin, William Muntean, Shu-chuan Kao
{"title":"A Hybrid Modified Angoff Standard Setting Method for Polytomous Items","authors":"Joe Betts, Cary Lin, William Muntean, Shu-chuan Kao","doi":"10.1111/emip.70044","DOIUrl":"https://doi.org/10.1111/emip.70044","url":null,"abstract":"<p>This research evaluates a Hybrid Angoff Method for standard setting procedures that utilize polytomous items. The method combines an Option Angoff with a Mean Angoff method to reduce the complexity of implementing judgments on polytomous items. The results were encouraging both with support for utilizing the method and panelist perceptions. The hybrid method provides a potential option for programs using polytomously scored items on their exams when implementing a standard setting panel.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"45 3","pages":""},"PeriodicalIF":2.2,"publicationDate":"2026-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148849100","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Autumn N. Wild M.A., Brian C. Leventhal Ph.D., Joseph M. Kush Ph.D.
{"title":"Uncovering the Landscape: A Closer Look at Internships in Educational Measurement","authors":"Autumn N. Wild M.A., Brian C. Leventhal Ph.D., Joseph M. Kush Ph.D.","doi":"10.1111/emip.70045","DOIUrl":"https://doi.org/10.1111/emip.70045","url":null,"abstract":"<p>High-impact practices such as internships have long been recognized for enhancing student learning and development. Yet, in educational measurement graduate training, internships remain underexamined. The current three-part study explored the structure, expectations, and implications of graduate-level internships in educational measurement and assessment. Part 1 reviewed 19 internship calls from summer 2024, identifying trends in required skills, application materials, and internship logistics. Part 2 surveyed 22 internship mentors and organizers to better understand specifics pertaining to mentorship, project supervision, the application review/interview process, and internship logistics. Part 3 surveyed 28 current graduate students, examining the value of internships from the intern perspective. Findings suggest that graduate internships in the fields of educational measurement and assessment are valuable and impactful experiences that serve a variety of stakeholders. Study limitations and directions of future research are discussed. Implications are provided for graduate students, graduate programs/faculty, internship organizers/mentors, and the field.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"45 3","pages":""},"PeriodicalIF":2.2,"publicationDate":"2026-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/emip.70045","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148784996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Metaphors, Mantras, and Mnemonics: Communication Competencies in Educational Measurement","authors":"Andrew Dean Ho","doi":"10.1111/emip.70039","DOIUrl":"https://doi.org/10.1111/emip.70039","url":null,"abstract":"<p>I provide evidence and arguments for increased investment in communication competencies in educational measurement. Building on the 2024 framework for Foundational Competencies in Educational Measurement, I emphasize communication as both a distinct domain and an interactive amplifier of collaboration, technical, and measurement competencies. I draw empirical and conceptual support from labor economics, psychometric job postings, and longstanding calls within measurement scholarship.</p><p>I then propose a conceptual toolkit called the “3–4–5 Triangle” of measurement mnemonics: the three Ws of test context, the four quadrants of testing purpose, and the five Cs of validity evidence. I illustrate this toolkit in teaching, research, and policy engagement, and situate it within broader commitments to collaboration, community engagement, and transformative research. I conclude by arguing that systematic cultivation and assessment of communication competencies are necessary for the continued relevance, impact, and legitimacy of educational measurement. This article is based on the address the author delivered as President of the National Council on Measurement in Education at the NCME Annual Meeting, April 25, 2025, Denver, Colorado. A video of the address as delivered is available at https://www.youtube.com/watch?v=QDQxkl9ypLU&t=143s.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"45 3","pages":""},"PeriodicalIF":2.2,"publicationDate":"2026-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148462929","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Qingshu Xu, Hong Jiao, Tianyi Zhou, Ming Li, Nan Zhang, Sydney Peters, Yanbin Fu
{"title":"Automated Alignment of Math Items to Content Standards in Large-Scale Assessments Using Language Models","authors":"Qingshu Xu, Hong Jiao, Tianyi Zhou, Ming Li, Nan Zhang, Sydney Peters, Yanbin Fu","doi":"10.1111/emip.70038","DOIUrl":"https://doi.org/10.1111/emip.70038","url":null,"abstract":"<p>Accurate alignment of items to content standards is critical for valid score interpretation in large-scale assessments. This study evaluates three automated paradigms for aligning Math items with four domain labels and nineteen skill labels. First, we extracted instruction-tuned embeddings, trained a suite of classical supervised machine-learning models, and examined whether principal component analysis (PCA) improved performance. Second, we fine-tuned a set of encoder-only language models (11 BERT-family models) for domain and skill alignment. Third, we explored ensemble learning via majority voting and stacking with multiple meta-models. Results show that DeBERTa-v3-base achieved the best weighted-average <i>F</i>1 score for domain alignment (0.950), whereas RoBERTa-large achieved the best weighted-average <i>F</i>1 score for skill alignment (0.869). PCA substantially improved linear classifiers based on embeddings but did not outperform fine-tuned encoder-only language models. Ensemble methods did not surpass the best single encoder-only language model. Overall, the study provides a systematic comparison of embedding-based classification, fine-tuned encoder-only language models, and ensemble strategies for automated item-to-standards alignment.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"45 3","pages":""},"PeriodicalIF":2.2,"publicationDate":"2026-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/emip.70038","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148462748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An Urgent Request for the Development of Classroom Assessment Literacy","authors":"Rick Stiggins","doi":"10.1111/emip.70032","DOIUrl":"https://doi.org/10.1111/emip.70032","url":null,"abstract":"<p>The author provides a retrospective on the evolution of his thinking about the role and power of classroom assessment based on four decades of collaboration with teachers. He contends that day-to-day classroom assessment can be far more than merely a way to hold students accountable for their learning. Rather, the research demonstrates that the assessment process and its results can be combined to create a powerful teaching and learning tool capable of building both students’ academic confidence and their achievement. He illustrates how teachers can partner with their students in ongoing classroom assessment processes that allow learners to track and gain a sense of control over their own academic success. This success, in turn, can give them the inner reserves of confidence needed to take the risk of continuing to strive to grow academically.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"45 3","pages":""},"PeriodicalIF":2.2,"publicationDate":"2026-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148466596","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An Educational Measurement Perspective on Dealing with Uncertainty and Bias in Classroom Formative Assessment","authors":"Randy E. Bennett","doi":"10.1111/emip.70031","DOIUrl":"https://doi.org/10.1111/emip.70031","url":null,"abstract":"<p>This article describes what educational measurement can contribute to the theory and practice of classroom formative assessment. Three contributions are highlighted. The first contribution, assessment as evidentiary reasoning, serves as the conceptual frame within which the other two contributions are elaborated. Those contributions are (1) uncertainty and strategies for its reduction and (2) bias with a focus on approaches to facilitating fairness. The foundational concept running through all three contributions is that of inference as to what students know and can do at a given time, the basis for taking next instructional steps. Such inferences may be strengthened when response consistency is observed across problem variations generated by the teacher or by the teacher in partnership with Large Language Models (LLMs).</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"45 3","pages":""},"PeriodicalIF":1.9,"publicationDate":"2026-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148153609","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Comparing Clustering Algorithms for Collusion Detection Using Mixed-Type Data","authors":"Soo Jeong Ingrisone, James Ingrisone","doi":"10.1111/emip.70034","DOIUrl":"https://doi.org/10.1111/emip.70034","url":null,"abstract":"<p>This study evaluates four clustering methodologies—Hierarchical Agglomerative Clustering (HAC), K-Prototypes, KAMILA, and Spectral Clustering—for detecting organized test collusion using mixed-type data. A two-phase design employed simulated data across 12 conditions and real certification exam data from a documented security breach. Results showed clustering efficacy increased substantially with higher exact response match rates and larger collusion group proportions. HAC emerged as the most robust method in simulations, while all four methods demonstrated exceptional convergence with empirically identified collusion groups in real data. These findings advance test security practice by providing empirically validated guidelines for selecting clustering methodologies and developing strategic implementation approaches, establishing a methodological foundation that equips testing organizations with enhanced collusion detection capabilities for evidence-based enforcement decisions.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"45 3","pages":""},"PeriodicalIF":1.9,"publicationDate":"2026-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148153610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evidence Centered Design: The Contribution of Bob Mislevy in Advancing Validity Theory","authors":"Drew H. Gitomer","doi":"10.1111/emip.70033","DOIUrl":"https://doi.org/10.1111/emip.70033","url":null,"abstract":"<p>Bob Mislevy's conception of evidence-centered design (ECD) is one of the major contributions to modern validity theory. Building on prior work in psychometrics, philosophy, legal reasoning, and statistics, ECD is a generative framework developed to support the principled design or redesign and scaling of both existing and emerging assessment practices. This paper provides an overview of the context that Mislevy saw as an opportunity to reimagine assessment design, some of the history of ECD development, including several seminal efforts. I examine some of the impacts of ECD and consider how ECD might evolve in the future.</p>","PeriodicalId":47345,"journal":{"name":"Educational Measurement-Issues and Practice","volume":"45 3","pages":""},"PeriodicalIF":1.9,"publicationDate":"2026-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/emip.70033","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148153585","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}