Descent along Nodal Straight Lines and Simplex Algorithm: Two Options of Regression Analysis Based on the Least Absolute Deviations Method

Abstract

A comparative analysis of the computational complexity of exact algorithms for estimating linear regression equations has been carried out using the least absolute deviations method. The aim of this study is to compare the computational efficiency of exact algorithms for descent along nodal straight lines and algorithms based on solving linear programming problems. To do that, the algorithm of gradient descent along nodal straight lines and algorithms for solving the equivalent primal and dual linear programming problems using the simplex method have been discussed. The computational complexity of the algorithms for implementing the least absolute deviation method in solving the primal and dual linear programming problems has been estimated. The average time for determining regression coefficients using the primal and dual linear programming problems and the average time for gradient descent along nodal straight lines have been compared in Monte Carlo statistical experiments. It is shown that both options are significantly inferior to the gradient descent along nodal straight lines in both the computational complexity of the algorithms and the computation time. The advantage of the algorithm for descent along nodal straight lines increases by two orders of magnitude or more with an increase in the sample size.