CHALLENGES AND OPPORTUNITIES FOR ESTIMATING EFFECTS WITH LARGE-SCALE EDUCATION DATA SETS

Abstract

School leaders and policy makers are often faced with serious challenges when determining the allocation of scarce resources for specific programs and practices. These decisions, typically made at the district, state or federal level, have become increasingly reliant on scientifically-based evidence that can inform what programs work, for whom, and under what conditions. To answer these questions researchers draw on a variety of methodological designs and statistical techniques to make robust inferences regarding the effect of relatively recent or existing programs and/or practices. The science of estimating effects has grown considerably over the past decade, aided in part by the availability of large-scale data sets that make it possible to simulate near-experimental conditions without employing traditional methods that require randomization of units (e.g., students, schools, districts) to treatment and control situations. These methods are particularly useful especially where randomization of subjects is not feasible. This article examines the opportunities and potential statistical problems when estimating effects with large-scale data sets for education policy and research.