Improving administrative data at scale: Experimental evidence on digital testing in Indian schools

Abstract

Large-scale student assessments are a cornerstone of proposed educational reforms to improve student achievement from very low levels in low and middle-income countries. Yet, this promise relies on their presumed reliability. I use direct audit evidence from a large Indian state (Andhra Pradesh) to show that, as currently administered, official learning assessments substantially overstate proficiency and understate the ‘learning crisis’ of low student achievement. In an experiment covering over 2400 schools, I evaluate whether digital tablet-based testing could reduce distortion. Although paper-based assessments proctored by teachers severely exaggerate achievement, tablet-based assessments closely match independent test data and are much less likely to be flagged for cheating. Further, I use the direct audit-based retest to directly validate of existing (indirect) statistical procedures for detecting cheating at scale and establish that it would be feasible to monitor data integrity cheaply and at scale with such methods. Overall, these results suggest that well-designed technology-aided interventions may improve data integrity at scale, without which these learning assessments are unlikely to serve as a catalyst for policy action.