Protocol for the pragmatic evaluation of the Department of the Air Force’s Personal Financial Readiness Program

Abstract

Federal policy mandates that United States military members receive numerous financial trainings across their career to improve their financial literacy and equip them to make informed financial decisions that, ultimately, promote positive individual and family well-being. Connected to this mandate, systematic evaluation efforts of the Department of the Air Force’s Personal Financial Readiness program were initiated to identify program outcomes and downstream results while simultaneously informing program improvements and modifications. When situated in a discussion of evaluation theory and design practices, documenting these outcome and process evaluation protocols can provide valuable insight for scientist-practitioners designing multi-site program evaluations in the context of real-world constraints, as is often the case for large institutions, including the military. The evaluation design includes a prospective cohort design for the outcome evaluation with surveys completed before and after training and 3- and 6-months following training. The surveys provide data on short-, mid-, and long-term outcomes. Data collected for the process evaluation, include targeted surveys and interviews with individuals conducting and participating in the training. Together, this protocol illustrates a theory-informed evaluation design of a complex, multi-site set of trainings utilizing diverse data collection methods to conduct a rigorous outcome and process evaluation.