CoChat

Derivative-free optimization: a review of algorithms and comparison of software implementations

Open access
Rios, Luis Miguel, Sahinidis, Nikolaos V. · 2013 · Journal of Global Optimization

The DOI matches its Crossref record; 130 of the 132 references it cites were found in a registry or an index.

Abstract

This paper addresses the solution of bound-constrained optimization problems using algorithms that require only the availability of objective function values but no derivative information. We refer to these algorithms as derivative-free algorithms. Fueled by a growing number of applications in science and engineering, the development of derivative-free optimization algorithms has long been studied, and it has found renewed interest in recent time. Along with many derivative-free algorithms, many software implementations have also appeared. The paper presents a review of derivative-free algorithms, followed by a systematic comparison of 22 related implementations using a test set of 502 problems. The test bed includes convex and nonconvex problems, smooth as well as nonsmooth problems. The algorithms were tested under the same conditions and ranked under several criteria, including their ability to find near-global solutions for nonconvex problems, improve a given starting point, and refine a near-optimal solution. A total of 112,448 problem instances were solved. We find that the ability of all these solvers to obtain good solutions diminishes with increasing problem size. For the problems used in this study, , , and are better, on average, than other derivative-free solvers in terms of solution quality within 2,500 function evaluations. These global solvers outperform local solvers even for convex problems. Finally, , , and show superior performance in terms of refining a near-optimal solution.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10898-012-9951-y

Cites

130 of 130 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Timeline of innovation in derivative-free optimization
Fig. 1 Timeline of innovation in derivative-free optimizationCC-BY · From the PDF
Image 2
CC-BY · From the PDF
Fig. 3 Number of variables versus number of test problems
Fig. 3 Number of variables versus number of test problemsCC-BY · From the PDF
Fig. 4 Fraction of problems over the 10 CPU minute limit
Fig. 4 Fraction of problems over the 10 CPU minute limitCC-BY · From the PDF
Fig. 5 Fraction of convex smooth problems solved as a function of allowable number of function evaluation
Fig. 5 Fraction of convex smooth problems solved as a function of allowable number of function evaluationCC-BY · From the PDF
Fig. 6 Fraction of convex nonsmooth problems solved as a function of allowable number of function evaluations
Fig. 6 Fraction of convex nonsmooth problems solved as a function of allowable number of function evaluationsCC-BY · From the PDF
Image 7
CC-BY · From the PDF
Image 8
CC-BY · From the PDF
Fig. 9 Fraction of nonconvex smooth problems solved as a function of allowable number of function evaluations
Fig. 9 Fraction of nonconvex smooth problems solved as a function of allowable number of function evaluationsCC-BY · From the PDF
Image 10
CC-BY · From the PDF
Image 11
CC-BY · From the PDF
Fig. 12 Fraction of nonconvex nonsmooth problems, as a function of allowable number of function evaluations, for which a solver found the best solution among all solvers tested
Fig. 12 Fraction of nonconvex nonsmooth problems, as a function of allowable number of function evaluations, for which a solver found the best solution among all solvers testedCC-BY · From the PDF
Image 13
CC-BY · From the PDF
Image 14
CC-BY · From the PDF
Fig. 15 Fraction of nonconvex smooth problems for which starting points were improved within 2,500 function evaluations vs. τ values
Fig. 15 Fraction of nonconvex smooth problems for which starting points were improved within 2,500 function evaluations vs. τ valuesCC-BY · From the PDF
Fig. 16 Fraction of nonconvex nonsmooth problems for which starting points were improved within 2,500 function evaluations vs. τ values
Fig. 16 Fraction of nonconvex nonsmooth problems for which starting points were improved within 2,500 function evaluations vs. τ valuesCC-BY · From the PDF
Fig. 17 Minimum number of solvers required to solve convex smooth test problems for various limits of function evaluations (best solver performance)
Fig. 17 Minimum number of solvers required to solve convex smooth test problems for various limits of function evaluations (best solver performance)CC-BY · From the PDF
Fig. 18 Minimum number of solvers required to solve convex nonsmooth test problems for various limits of function evaluations (best solver performance)
Fig. 18 Minimum number of solvers required to solve convex nonsmooth test problems for various limits of function evaluations (best solver performance)CC-BY · From the PDF
Fig. 19 Minimum number of solvers required to solve nonconvex smooth test problems for various limits of function evaluations (best solver performance)
Fig. 19 Minimum number of solvers required to solve nonconvex smooth test problems for various limits of function evaluations (best solver performance)CC-BY · From the PDF
Fig. 20 Minimum number of solvers required to solve nonconvex nonsmooth test problems for various limits of function evaluations (best solver performance)
Fig. 20 Minimum number of solvers required to solve nonconvex nonsmooth test problems for various limits of function evaluations (best solver performance)CC-BY · From the PDF
Fig. 21 Minimum number of solvers required to solve all test problems for various limits of function evaluations (best solver performance)
Fig. 21 Minimum number of solvers required to solve all test problems for various limits of function evaluations (best solver performance)CC-BY · From the PDF
Image 22
CC-BY · From the PDF
Fig. 23 Fraction of problems with three to nine variables that were solved
Fig. 23 Fraction of problems with three to nine variables that were solvedCC-BY · From the PDF
Fig. 24 Fraction of problems with 10-30 variables that were solved
Fig. 24 Fraction of problems with 10-30 variables that were solvedCC-BY · From the PDF
Fig. 25 Fraction of problems with 31-300 variables that were solved
Fig. 25 Fraction of problems with 31-300 variables that were solvedCC-BY · From the PDF
Fig. 26 Scaled results for the best, mean, median and worst result among the 10 optimization instances after 2,500 function evaluations for each solver
Fig. 26 Scaled results for the best, mean, median and worst result among the 10 optimization instances after 2,500 function evaluations for each solverCC-BY · From the PDF
Image 27
CC-BY · From the PDF
Image 28
CC-BY · From the PDF
Image 29
CC-BY · From the PDF
Image 30
CC-BY · From the PDF
Image 31
CC-BY · From the PDF
Image 32
CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 130 references

Research works printed but not linked: 9

  1. Awasthi, S.: Molecular Docking by Derivative-Free Optimization Solver. Master's thesis, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh (2008)
  2. Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Serafini, D.B., Torczon, V.J., Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Optim. 17, 1-13 (1999) · DOI · Paper page
  3. Chang, K.-F.: Modeling and Optimization of Polymerase Chain Reaction Using Derivative-Free Opti- mization. Master's thesis, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh (2011)
  4. Desai, R.: A Comparison of Algorithms for Optimizing the Omega Function. Master's thesis, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh (2010)
  5. Holland, J.H.: Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbor (1975)
  6. Powell, M.J.D.: A direct search optimization method that models the objective by quadratic interpolation. In: Presentation at the 5th Stockholm Optimization Days (1994) 111. · DOI · Paper page
  7. Wang, H.: Application of Derivative-Free Algorithms in Powder Diffraction. Master's thesis, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh (2011) 140.
  8. Winfield, D.: Function and Functional Optimization by Interpolation in Data Tables. PhD thesis, Harvard University, Cambridge (1969) 142.
  9. Zheng, Y.: Pairs Trading and Portfolio Optimization. Master's thesis, Department of Chemical Engi- neering, Carnegie Mellon University, Pittsburgh (2011)

Cited by

See all 1,276

Metadata

Authors
Rios, L. M., & Sahinidis, N. V.
Affiliations
Carnegie Mellon University (US)
Corresponding author
Sahinidis, Nikolaos V.
Venue
Journal of Global Optimization
Published
2012-07-12
Volume
56
Issue
3
Pages
1247-1293
ISSN
  • 0925-5001 (ISSN-L)
  • 1573-2916
OpenAlex
W2160960847
Cited by
1,276
References
132
Citations by year
2026: 57 · 2025: 65 · 2024: 79 · 2023: 99 · 2022: 114 · 2021: 138 · 2020: 167 · 2019: 125 · 2018: 108 · 2017: 93 · 2016: 87 · 2015: 57 · 2014: 58 · 2013: 15 · 2012: 8
FWCI
52.92
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Global Optimization (publishedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
Funders
  • National Science Foundation (1033661, DE-FE-0004000)
  • National Institutes of Health
  • National Energy Technology Laboratory (DE-FE-0004000, 0004000)
  • National Institute of General Medical Sciences (GM072023)
  • Division of Chemical, Bioengineering, Environmental, and Transport Systems
Topics
  • Physical Sciences › Mathematics › Numerical Analysis › Advanced Optimization Algorithms Research
  • Physical Sciences › Computer Science › Artificial Intelligence › Metaheuristic Optimization Algorithms Research
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Optimization and Variational Analysis
Keywords
derivative-free optimization, bound-constrained optimization, global optimization, local optimization, solver comparison, non-convex optimization, nonsmooth optimization, convex optimization, solution refinement, function evaluations

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Derivative-free optimization: a review of algorithms and comparison of software implementations. https://app.cochat.ai/papers/10.1007/s10898-012-9951-y
The paper
Rios, L. M., & Sahinidis, N. V. (2013). Derivative-free optimization: a review of algorithms and comparison of software implementations. Journal of Global Optimization. https://doi.org/10.1007/s10898-012-9951-y

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.