Measuring the impact of academic literacy interventions: Refining an evaluation design through self-reflection and feedback

Abstract

This article, located in the discipline of academic literacy studies, draws upon the fields of critical realism, design research, and evaluation studies. It reports on the validation of a flexible evaluation design for assessing the impact of academic literacy interventions. The design was validated in two ways. Firstly, through a process of critical reflection, the researcher considers her own experience with applying the evaluation design to an academic literacy course; the weaknesses and limitations that emerged from this implementation are considered. Secondly, academic literacy specialists responsible for a wide variety of interventions in South Africa were consulted by means of a questionnaire containing both quantitative and qualitative questions. The purpose of this questionnaire was to determine to which extent the evaluation design could be applied to a variety of academic literacy interventions in various contexts. Recommendations regarding the refinement of the evaluation design are made, and a revised evaluation design is put forward.