Targeted assessment of hypothesis testing skills using cognitive diagnostic models: Implications for formative practice

Abstract

Formative assessment is essential for identifying student learning gaps and supporting meaningful feedback, particularly in subjects that require multi-step reasoning such as statistical hypothesis testing. This study demonstrates how Cognitive Diagnostic Models (CDMs) can enhance assessment practices by offering detailed, attribute-level feedback on student proficiency. Analyzing item response data from 219 undergraduate students in an introductory statistics course, the study employed the Generalized DINA model and its reduced variants, identifying the most suitable and interpretable model. Through expert evaluation and Q-matrix validation procedures, four attributes involved in hypothesis testing were specified and refined. Among the six models, the Linear Logistic Model (LLM) yielded the best fit.The attribute classification results revealed that while most students mastered procedural aspects of hypothesistesting, distinct groups struggled either with selecting appropriate statistical methods or managing multi-step computations. The study underscores the potential of CDM-based assessments to provide actionable diagnostic information for tailored instruction, targeted feedback, and pinpointing specific learning hurdles. While this approach is readily applicable in large classes, its effectiveness can also extend to smaller groups by aggregating data across multiple cohorts. CDMs offer a flexible and scalable framework for improving assessment-for-learning practices across structured subject areas.