Contributing to monitoring and evaluation practices by conceptualizing a Valuable Insights from Log Analysis (VILA) Framework

Abstract

Log analyses are proposed in response to the evidence base required for log-generating Apps. This article presents the VILA (Valuable Insights from Log Analysis) Framework, a conceptual framework that structures how valuable insights can be generated from log analysis to support Monitoring and Evaluation (M&E) practices. The design, development, and evaluation process is centered in the pragmatic research paradigm and follows the Design Science Research (DSR) methodology. Existing literature was reviewed to formulate building blocks and design requirements for the VILA Framework. The VILA Framework was evaluated and applied through expert reviews and an industry case study. Relevance cycles assessed and refined the VILA Framework. This article presents the main concepts, categories, and sub-concepts to derive valuable insights towards potential improvements of log-generating Apps. The VILA Framework focused on the quantitative log data concepts; however, it acknowledges the essential qualitative concepts needed to derive value from the log data. Despite its limitations, the VILA Framework provides a starting point for researchers to structure and implement log analyses beyond descriptive statistics towards increased or improved M&E practices.