A Replication of Comparative Study of Moving Windows on Linear Regression and Estimation by Analogy

Abstract

Context: Recent studies have shown that estimation accuracy can be affected by only using a window of recent projects as training data for building an effort estimation model. The effect and its extent can be affected by effort estimation methods (e.g. linear regression (LR) or estimation by analogy (EbA)), windowing policies (fixed-size or fixed-duration), and between organizations. However, different effects between organizations have only been explored with LR as the estimation method, and different effects between estimation methods and windowing policies have mainly been explored with data from only one organization. Objective: To further investigate the effect on estimation accuracy of using windows, with different windowing policies, when using EbA as the estimation method. Also, to compare the effect of LR with EbA as an estimation method, when using windows. Method: Using a data set studied with LR in previous research, we examine the effects of using windows on the accuracy of effort estimates, using EbA with both fixed-size and fixed-duration windowing policies. Results: With this data set, fixed-size windows, no matter their size, do not improve the accuracy of estimates obtained using EbA. This reinforces previous research with this data set, which used LR as the estimation approach. However, fixed-duration windows can improve the accuracy of estimates obtained with EbA. This contradicts previous research with this data set, which used LR as the estimation approach. Variations in the settings for EbA can change the sizes at which windows are helpful. Conclusions: This study reinforces that the effect of using windows can be affected by the effort estimation approach, and by the windowing policy. Contrary to previous research, fixed-duration windows are found to be more helpful than fixed-size windows, and significant improvements are found with EbA that were not found with LR. Further research is needed to understand these differences.