A revised stochastic nelder-mead algorithm for numerical optimization

Abstract

The Stochastic Nelder-Mead, a recently developed variant of the classic Nelder-Mead algorithm, is a direct search method for derivative-free, nonlinear and black-box stochastic optimization problem. A key factor that influences its performance is obtaining reasonable rankings on the simplex points with random noise. We propose a new ranking procedure that integrates a selection sorting algorithm with statistical hypothesis testing method. This procedure provides an efficient `fine-granular' re-sampling scheme in which the sample sizes can be estimated more precisely and with more flexibility. A numerical study indicates that the revised algorithm can generally outperform its original in terms of both accuracy and stability.