Basin of Attraction as a measure of robustness of an optimization algorithm

Abstract

The concept of Basin of Attraction (BOA)from the theory of dynamical systems can be applied to evaluate the robustness of a deterministic optimization algorithm. For an objective function with many local minima, a large BOA with smooth boundaries associated with the global minimum is an important indicator for the robustness of the optimization algorithm. In this paper, numerical examples of BOA for canned commercial optimizer: fmincon in MATLAB's toolbox (Sequential Quadratic Programming, sqp, and Interior-Point Algorithm)are given as illustrations of how BOA can be used as a tool to compare the robustness of optimization algorithms. We also showed in an example of machine learning application, spurious local minima often appear with more training data are added, and these spurious local minima have nothing to do with the legitimate solution. Finally, three different types of quantitative measure of the robustness of an optimization algorithm based on the basin boundaries are proposed.