Comparisons between Resampling Techniques in Linear Regression: A Simulation Study

Abstract

The classic methods used in estimating the parameters in linear regression need to fulfill some assumptions. If the assumptions are not fulfilled, the conclusion is questionable. Resampling is one of the ways to avoid such problems. The study aims to compare resampling techniques in linear regression. The original data used in the study is clean, without any influential observations, outliers and leverage points.  The ordinary least square method was used as the primary method to estimate the parameters and then compared with resampling techniques. The variance, p-value, bias, and standard error are used as a scale to estimate the best method among random bootstrap, residual bootstrap and delete-one Jackknife. After all the analysis took place, it was found that random bootstrap did not perform well while residual and delete-one Jackknife works quite well. Random bootstrap, residual bootstrap, and Jackknife estimate better than ordinary least square. Is was found that residual bootstrap works well in estimating the parameter in the small sample. At the same time, it is suggested to use Jackknife when the sample size is big because Jackknife is more accessible to apply than residual bootstrap and Jackknife works well when the sample size is big.