Introduction to Derivative-Free Optimization

Abstract

The absence of derivatives, often combined with the presence of noise or lack of smoothness, is a major challenge for optimization. This book explains how sampling and model techniques are used in derivative-free methods and how these methods are designed to efficiently and rigorously solve optimization problems. Although readily accessible to readers with a modest background in computational mathematics, it is also intended to be of interest to researchers in the field. Introduction to Derivative-Free Optimization is the first contemporary comprehensive treatment of optimization without derivatives. This book covers most of the relevant classes of algorithms from direct search to model-based approaches. It contains a comprehensive description of the sampling and modeling tools needed for derivative-free optimization; these tools allow the reader to better understand the convergent properties of the algorithms and identify their differences and similarities. Introduction to Derivative-Free Optimization also contains analysis of convergence for modified Nelder Mead and implicit-filtering methods, as well as for model-based methods such as wedge methods and methods based on minimum-norm Frobenius models. Audience: The book is intended for anyone interested in using optimization on problems where derivatives are difficult or impossible to obtain. Such audiences include chemical, mechanical, aeronautical, and electrical engineers, as well as economists, statisticians, operations researchers, management scientists, biological and medical researchers, and computer scientists. It is also appropriate for use in an advanced undergraduate or early graduate-level course on optimization for students having a background in calculus, linear algebra, and numerical analysis. Contents: Preface; Chapter 1: Introduction; Part I: Sampling and modeling; Chapter 2: Sampling and linear models; Chapter 3: Interpolating nonlinear models; Chapter 4: Regression nonlinear models; Chapter 5: Underdetermined interpolating models; Chapter 6: Ensuring well poisedness and suitable derivative-free models; Part II: Frameworks and algorithms; Chapter 7: Directional direct-search methods; Chapter 8: Simplicial direct-search methods; Chapter 9: Line-search methods based on simplex derivatives; Chapter 10: Trust-region methods based on derivative-free models; Chapter 11: Trust-region interpolation-based methods; Part III: Review of other topics; Chapter 12: Review of surrogate model management; Chapter 13: Review of constrained and other extensions to derivative-free optimization; Appendix: Software for derivative-free optimization; Bibliography; Index.