Predicting the Vector Impact of Change - An Industrial Case Study at Brightsquid

Abstract

Background: Understanding and controlling the impact of change decides about the success or failure of evolving products. The problem magnifies for start-ups operating with limited resources. Their usual focus is on Minimum Viable Product (MVP's) providing specialized functionality, thus have little expense available for handling changes. Aims: Change Impact Analysis (CIA) refers to the identification of source code files impacted when implementing a change request. We extend this question to predict not only affected files, but also the effort needed for implementing the change, and the duration necessary for that. Method: This study evaluates the performance of three textual similarity techniques for CIA based on Bag of words in combination with either topic modeling or file coupling. Results: The approaches are applied on data from two industrial projects. The data comes as part of an industrial collaboration project with Brightsquid, a Canadian start-up company specializing in secure communication solutions. Performance analysis shows that combining textual similarity with file coupling improves impact prediction, resulting in Recall of 67%. Effort and duration can be predicted with 84% and 72% accuracy using textual similarity only. Conclusions: The relative effort invested into CIA for predicting impacted files can be reduced by extending its applicability to multiple dimensions which include impacted files, effort, and duration.