MEASURING TREATMENT EFFECTS OF ONLINE VIDEOS ON ACADEMIC PERFORMANCE USING DIFFERENCE-IN-DIFFERENCES ESTIMATIONS

Abstract

Supplementing student learning with online videos has produced mixed results in respect of improving academic performance. This study proposed that these mixed results, before COVID-19, are due to most of the literature on online videos being observational studies and not taking confounding factors into account. This study applied the difference-in-differences (DID) technique, which measures treatment effects in observational studies. Using student grades data from an engineering mechanics course, the treatment effects of videos were measured 1) on the entire group that used the videos, 2) on those initially with failing and then with passing grades, and 3) on those grouped by grade percentage ranges. It was found that the videos had no effect on the entire group but did significantly affect those who had initial failing grades — specifically, grades of 40% to 49%. A main finding was that initial grades are an indication of how effective the online videos are in improving grades.