How Can We Accurately Measure Whether Students are Gaining Relevant Outcomes in Higher Education?

The main objective of this study is to empirically test a number of theory-based models (i.e. fixed effects (FE), random effects (RE), and aggregated residuals (AR)) to measure both, the generic knowledge as well as the degree attainment rates and early labor outcomes, gained by students in different programs and institutions in higher education. There are four main findings: First, the results of the paper confirm the need of using models that address the issue of student selection into programs and institutions in order to avoid biased estimates. Second, our findings provide suggestive evidence in favor of using FE models. Third, the results also illustrate the need to use appropriate statistical corrections (e.g., Heckman type selection models) to also address the issue related to students dropping out of college. Finally, our findings confirm our hypotheses that rankings of specific college-program combinations change depending on different educational and labor outcome measures considered. This finding emphasizes the need to use complementary indicators related to the mission of the specific post-secondary institutions that are being ranked. The results of this paper illustrate the importance of validating empirical models intended to rank college-program contributions according to a number of educational and early labor market outcomes. Finally, given the sensitivity of the models to different model specifications, it is not clear that they should be used to make any high-stakes decisions in higher education. They could, however, serve as part of a broader set of indicators to support programs and colleges as part of a formative evaluation.