Association of Defect Log Suitability for Machine Learning with Performance: An Experience Report

Abstract

Machine learning (ML) based solutions utilizing textual details in defect logs have been shown to enable automation of defect management process and make it cost effective. In this work, we assess effectiveness of apriori manual analysis of the suitability of applying ML to problems encountered during defect management process. We consider problems of mapping defects to service engineers and business processes for designing experiments. Experimental analysis on these problems using multiple defect logs from practice reveals that a systematic analysis of the defect log data by project experts can provide approximate indication of the eventual performance of the ML model even before they are actually built. We discuss practical significance of the conclusions for designing ML based solutions in-practice.