Staggered Versus All-At-Once Content Release in Massive Open Online Courses: Evaluating a Natural Experiment

Abstract

We report on an experiment testing the effects of releasing all of the content in a Massive Open Online Course (MOOC) at launch versus in a staggered release. In 2013, HarvardX offered two "runs" of the HeroesX course: In the first, content was released weekly over four months; in the second, all content was released at once. We develop three operationalizations of "ontrackness" to measure how students participated in sync with the recommended syllabus. Ontrackness in both versions was low, though in the second, mean ontrackness was approximately one-half of levels in the first HeroesX. We find few differences in persistence, participation, and completion between the two runs. Controlling for a students' number of active weeks, we estimate modest positive effects of ontrackness on certification. The revealed preferences of students for flexibility and the minimal benefits of ontrackness suggest that releasing content all at once may be a viable strategy for MOOC designers.