Evaluating single-best-answer questions in dental materials education: a retrospective item analysis from an online undergraduate dentistry question bank.

Abstract

Background: High-quality single-best-answer (SBA) questions require appropriate construction, alignment with intended learning outcomes, absence of technical item flaws and psychometric evaluation. Dentistry-specific evidence linking these constructional features to item performance remains limited. This study analysed Dental Materials responses from DentTest, an online undergraduate dentistry question bank, to identify item characteristics associated with higher discrimination.

Methods: A retrospective learning-analytics item analysis was performed using first-attempt response data. At extraction, 134 users were eligible, of whom 76 provided research consent (56.7%). The dataset contained 44,964 user×item observations from 76 users across 1560 Dental Materials items. Primary analyses were restricted to 691 items with ≥30 unique first-attempt users. Analyses included item psychometric parameters (facility, discrimination index, number of functioning distractors and distractor entropy) and selected constructional features (option-length cueing, stem word count and negative-stem wording).

Results: Discrimination index was inversely correlated with facility (r = -0.471; 95% CI -0.527 to -0.411). In the multivariable quadratic model, the estimated peak discrimination occurred at p = 0.578, consistent with an inverted-U relationship between facility and discrimination. Functioning distractors (r = 0.531; 95% CI 0.475-0.582) and distractor entropy (r = 0.471; 95% CI 0.411-0.527) were positively associated with discrimination index. However, split-half stability was low (r = 0.15), with 542/691 items (78.4%) changing discrimination band at least once across repeated splits.

Conclusion: In this formative dentistry question bank, Dental Materials SBAs with higher discrimination were associated with moderate facility, stronger distractor functioning and more informative distractor-choice patterns. Item analytics should be viewed as an early-warning system rather than a final judgement, guiding educators towards items that require revision.