The future of large-scale experiments and their challenges in the digital era.

Abstract

Large-scale randomized experiments involving millions of observations have transformed social science across academic and applied domains. These methods are now widely used in fields ranging from economics and political science to public health, education and digital product development. Drawing on the experience of industry practitioners and academic collaborators, this Perspective outlines six areas where the expanding scale and scope of experimental studies have introduced new challenges. We describe open problems related to organizational incentives and experimental governance; privacy, fairness and ethics; long-term impact estimation; time-adaptive experimental studies; heterogeneous treatment effects; and generative artificial intelligence. For each area, we summarize recent methodological advances, highlight ongoing limitations and suggest directions for future work. Our goal is to surface practical challenges that merit greater attention from the research community and to encourage sustained collaboration between academics and practitioners in shaping the future of experimental studies.