Convergent variants of the Nelder-Mead algorithm.
Degree GrantorUniversity of Canterbury
Degree NameMaster of Science
The Nelder-Mead algorithm for unconstrained optimisation has been used extensively to solve parameter estimation and other problems since its inception in 1965. Despite its age it is still the method of choice for many practitioners in the fields of statistics, engineering and the physical and medical sciences because it is easy to code and very easy to use. It belongs to the direct search class of methods which do not require derivatives and which are often claimed to be robust for problems with discontinuities or where the function values are affected by noise. Recently (1998), McKinnon has shown that that the method can converge to non-solutions for certain classes of problems. Only very limited convergence results exist (Lagarias et al) for a restricted class of problems in one or two dimensions. An overview of selected direct search methods for unconstrained optimisation is given and several variants of the Nelder-Mead algorithm are presented, culminating in a variant which is provably convergent in any number of dimensions, and which performs well in practice.