Numerical Optimization

Abstract

This chapter presents methods for numerically determining the maximum likelihood estimation (m.l.e.) and their computer implementations. The existence of a closed-form solution to the m.l.e. is far more the exception than the rule: it is often defined only implicitly, and numerical methods will be necessary to locate it. When the usual regularity conditions are satisfied, the m.l.e. can be obtained either by finding the roots of the score function, or by directly maximizing the likelihood function using numerical optimization algorithms. The chapter discusses evolutionary algorithm designation as a general approach that implies that it is suited for the optimization of possibly nonconvex, nonsmooth or even noncontinuous functions, precisely the situation in which gradient-and Hessian-based methods (two general thumb rules followed in numerical m.l.e.) are prone to fail. It explains two classes of evolutionary algorithms: genetic algorithms (differential evolution algorithm) and evolutionary strategies (Covariance Matrix Adaption Evolutionary Strategy algorithm).