Improving validity and efficiency of digital dyslexia screening through trial-by-trial feedback.

Abstract

Trial-by-trial feedback is a powerful tool across scientific disciplines, offering real-time information that shapes learning, attention, and behavioral adjustment. However, its role in large-scale educational and psychological assessments remains underexplored. This study examines whether providing trial-by-trial performance feedback during a digital dyslexia screener enhances engagement without compromising score validity or interpretability. Engagement is a critical threat to validity in assessments, particularly among young children in digital universal screening contexts. We conducted two large-scale experiments involving 6,332 students from Grades 1 to 12 in Colombia and the United States in which students were randomly assigned to one of two conditions: (a) informative feedback (auditory cues indicating correct/incorrect) or (b) non-informative feedback (neutral sound). The assessment was administered in two formats: an adaptively ordered English version and a randomly ordered Spanish version. We evaluated differences in response behaviors, item response theory model fit, and concurrent validity between conditions. Results indicate that informative feedback significantly increased compliance and reduced disengagement. Assessment scores and item response theory model fit were comparable across conditions. Notably, informative feedback improved the concurrent validity of scores and was associated with shorter completion times. These findings suggest that trial-by-trial feedback can enhance engagement while preserving score interpretability, leading to more efficient and valid digital universal screening assessments. (PsycInfo Database Record (c) 2026 APA, all rights reserved).