Exploiting a natural experiment in assessing student learning outcomes in public and nonprofit administration: A demonstration

Abstract

This study exploits a natural experiment to establish the equivalence and/or difference in student learning outcomes between online and face-to-face public and nonprofit administration courses. Its main contribution is thus methodological. We will reexamine the online v. classroom debate - the face-to-face lecture is still the most effective method to deliver course content to students - using a new dataset and estimation approach. Our research addresses this causal inference question: Does the format of course delivery impact student learning? The most robust empirical strategy to rule out alternative explanations in causal studies is the experimental approach. In this study, we did not employ the experimental research design or any standard techniques, for example, regression analysis, available to the program evaluator or policy analyst. Instead, we exploited a naturally occurring phenomenon in a classroom environment to approximate statistical equivalence in the characteristics of students in the online and classroom formats and satisfy the exogeneity assumption of the treatment variable. Its more practical contribution is the use of learning theory and new research in online pedagogy to discuss the study’s conclusions and implications for online programming, instruction, and program coordination. We developed the feedback as teaching philosophy or approach to close the gap between the learning outcomes of completely asynchronous online and entirely face-to-face classes in public administration.