Industrial Evaluation of a Log File Analysis Methodology

Abstract

Test result evaluation programs often take the form of log file analyzers, which analyze text logs of events that have happened during testing. Previously, we proposed a methodology for deriving logging instrumentation and state-based log file analyzer programs from requirements. In this paper, we report on an industrial evaluation in which the methodology was carried out on two pieces of commercial software in a commercial setting. We report on our quantitative and qualitative observations and on recommendations for improving the methodology and the tools used.