The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Iterative Methods for Optimization
Abstract
This book presents a carefully selected group of methods for unconstrained and bound constrained optimization problems and analyzes them in depth both theoretically and algorithmically. It focuses on clarity in algorithmic description and analysis rather than generality, and while it provides pointers to the literature for the most general theoretical results and robust software, the author thinks it is more important that readers have a complete understanding of special cases that convey essential ideas. A companion to Kelley's book, Iterative Methods for Linear and Nonlinear Equations (SIAM, 1995), this book contains many exercises and examples and can be used as a text, a tutorial for self-study, or a reference. Iterative Methods for Optimization does more than cover traditional gradient-based optimization: it is the first book to treat sampling methods, including the Hooke& Jeeves, implicit filtering, MDS, and Nelder& Mead schemes in a unified way.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Optimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
An Algorithm for Least-Squares Estimation of Nonlinear Parameters
Marquardt, D. W. (1963) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0111030 · Cited by 30,728
A Simplex Method for Function Minimization
Nelder, J. A., & Mead, R. (1965) · The Computer Journal · doi:10.1093/comjnl/7.4.308 · Cited by 28,905 · cited 6×
“Restarting the Nelder–Mead Algorithm . . . . . . . . . . . . . . . .”
InfluentialRudin, W. (1973) · Cited by 13,740
A method for the solution of certain non-linear problems in least squares
Levenberg, K. (1944) · Quarterly of Applied Mathematics · doi:10.1090/qam/10666 · Cited by 12,359
Iterative Solution of Nonlinear Equations in Several Variables
Ortega, J. M., & Rheinboldt, W. C. (2000) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898719468 · Cited by 6,860
The Levenberg-Marquardt algorithm: Implementation and theory
Moré, J. J. (1978) · Lecture Notes in Mathematics · doi:10.1007/bfb0067700 · Cited by 5,773
Principles of mathematical analysis
Rudin, W. (1964) · UCLA - Biblioteca de Ciencias y Tecnologia Felix Morales Bueno · Cited by 5,300
Function minimization by conjugate gradients
Fletcher, R. (1964) · The Computer Journal · doi:10.1093/comjnl/7.2.149 · Cited by 4,939
Linear and nonlinear programming.
(1986) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(86)90092-3 · Cited by 4,805
A Rapidly Convergent Descent Method for Minimization
Fletcher, R., & Powell, M. J. D. (1963) · The Computer Journal · doi:10.1093/comjnl/6.2.163 · Cited by 4,583
A generalized inverse for matrices
Penrose, R. (1955) · Mathematical Proceedings of the Cambridge Philosophical Society · doi:10.1017/s0305004100030401 · Cited by 4,551
`` Direct Search'' Solution of Numerical and Statistical Problems
Hooke, R., & Jeeves, T. A. (1961) · Journal of the ACM · doi:10.1145/321062.321069 · Cited by 4,254
Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales
Banach, S. (1922) · Fundamenta Mathematicae · doi:10.4064/fm-3-1-133-181 · Cited by 4,013
Numerical Initial Value Problems in Ordinary Differential Equations
I., E., & Gear, C. W. (1973) · Mathematics of Computation · doi:10.2307/2005674 · Cited by 3,926
Conditioning of quasi-Newton methods for function minimization
Shanno, D. F. (1970) · Mathematics of Computation · doi:10.1090/s0025-5718-1970-0274029-x · Cited by 3,872
Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods
Barrett, R., Berry, M., Chan, T. F., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., & van der Vorst, H. (1994) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611971538 · Cited by 3,479
Numerical Solution of Initial-Value Problems in Differential-Algebraic Equations
Brenan, K. E., Campbell, S. L., & Petzold, L. R. (1995) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611971224 · Cited by 3,405
A family of variable-metric methods derived by variational means
Goldfarb, D. (1970) · Mathematics of Computation · doi:10.1090/s0025-5718-1970-0258249-6 · Cited by 3,325
Updating quasi-Newton matrices with limited storage
Nocedal, J. (1980) · Mathematics of Computation · doi:10.1090/s0025-5718-1980-0572855-7 · Cited by 2,958
A class of methods for solving nonlinear simultaneous equations
Broyden, C. G. (1965) · Mathematics of Computation · doi:10.1090/s0025-5718-1965-0198670-6 · Cited by 2,788
Introduction to matrix computations
Stewart, G. W. (1973) · International Linear Collider · Cited by 2,381
Nonlinear Programming: Sequential Unconstrained Minimization Techniques
Fiacco, A. V., & McCormick, G. P. (1968) · Defense Technical Information Center (DTIC) · Cited by 2,262
Minimization of functions having Lipschitz continuous first partial derivatives
Armijo, L. (1966) · Pacific Journal of Mathematics · doi:10.2140/pjm.1966.16.1 · Cited by 1,983
Generalized Inverses of Linear Transformations
Campbell, S. L., & Meyer, C. D. (2009) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898719048 · Cited by 1,897
Foundations of Optimal Control Theory.
Wishart, D. M. G., Lee, E. B., & Markus, L. (1969) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2343766 · Cited by 1,862
Dembo, R. S., Eisenstat, S. C., & Steihaug, T. (1982) · SIAM Journal on Numerical Analysis · doi:10.1137/0719025 · Cited by 1,793
Numerical Methods for Nonlinear Variational Problems
Glowinski, R. (1984) · Springer eBooks · doi:10.1007/978-3-662-12613-4 · Cited by 1,728
Numerical Methods for Nonlinear Variational Problems
Glowinski, R., & Oden, J. T. (1985) · Journal of Applied Mechanics · doi:10.1115/1.3169136 · Cited by 1,541
Moré, J. J., & Sorensen, D. C. (1983) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0904038 · Cited by 1,411
Adjustment of an Inverse Matrix Corresponding to a Change in One Element of a Given Matrix
Sherman, J., & Morrison, W. J. (1950) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729893 · Cited by 1,402
Sequential Application of Simplex Designs in Optimisation and Evolutionary Operation
Spendley, W., Hext, G. R., & Himsworth, F. R. (1962) · Technometrics · doi:10.1080/00401706.1962.10490033 · Cited by 1,379
Genetic Algorithms and the Optimal Allocation of Trials
Holland, J. H. (1973) · SIAM Journal on Computing · doi:10.1137/0202009 · Cited by 1,239
The conjugate gradient method in extremal problems
Polyak, B. T. (1969) · USSR Computational Mathematics and Mathematical Physics · doi:10.1016/0041-5553(69)90035-4 · Cited by 1,164
Convergence Conditions for Ascent Methods
Wolfe, P. (1969) · SIAM Review · doi:10.1137/1011036 · Cited by 1,106
Iterative Methods for Solving Linear Systems
(2006) · Texts in Applied Mathematics · doi:10.1007/0-387-22750-4_4 · Cited by 1,014
Note sur la convergence de méthodes de directions conjuguées
Polak, E., & Ribiere, G. (1969) · Revue française d'informatique et de recherche opérationnelle. Série rouge · doi:10.1051/m2an/196903r100351 · Cited by 986
Description of DASSL: a differential/algebraic system solver
Petzold, L. (1982) · University of North Texas Digital Library (University of North Texas) · Cited by 968
An Adaptive Nonlinear Least-Squares Algorithm
Dennis, J. E., Gay, D. M., & Walsh, R. E. (1981) · ACM Transactions on Mathematical Software · doi:10.1145/355958.355965 · Cited by 930
The Conjugate Gradient Method and Trust Regions in Large Scale Optimization
Steihaug, T. (1983) · SIAM Journal on Numerical Analysis · doi:10.1137/0720042 · Cited by 824
GOLUBf, G. H., & Pereyra, V. L. (1973) · Cited by 732
Projected Newton Methods for Optimization Problems with Simple Constraints
Bertsekas, D. P. (1982) · SIAM Journal on Control and Optimization · doi:10.1137/0320018 · Cited by 613
Projected gradient methods for linearly constrained problems
Calamai, P. H., & Moré, J. J. (1987) · Mathematical Programming · doi:10.1007/bf02592073 · Cited by 588
On the Goldstein-Levitin-Polyak gradient projection method
Bertsekas, D. P. (1976) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1976.1101194 · Cited by 583
Newton’s Method with a Model Trust Region Modification
Sorensen, D. C. (1982) · SIAM Journal on Numerical Analysis · doi:10.1137/0719026 · Cited by 560
On the Local and Superlinear Convergence of Quasi-Newton Methods
BROYDEN, C. G., DENNIS, J. E., & MORÉ, J. J. (1973) · IMA Journal of Applied Mathematics · doi:10.1093/imamat/12.3.223 · Cited by 502
Estimation of Sparse Jacobian Matrices and Graph Coloring Blems
Coleman, T. F., & Moré, J. J. (1983) · SIAM Journal on Numerical Analysis · doi:10.1137/0720013 · Cited by 468
Structure and binding of Lennard-Jones clusters: 13≤N≤147
Northby, J. A. (1987) · The Journal of Chemical Physics · doi:10.1063/1.453492 · Cited by 467
Concus, P., Golub, G. H., & O'Leary, D. P. (1976) · Sparse Matrix Computations · doi:10.1016/b978-0-12-141050-6.50023-4 · Cited by 433
ON THE SOLUTION OF NONLINEAR FINITE ELEMENT EQUATIONS.
Bathe, K., Dvorkin, E. N., & Kojić, M. (1984) · Cited by 430
An Inverse Matrix Adjustment Arising in Discriminant Analysis
Bartlett, M. S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729698 · Cited by 429
Truncated-Newton algorithms for large-scale unconstrained optimization
Dembo, R. S., & Steihaug, T. (1983) · Mathematical Programming · doi:10.1007/bf02592055 · Cited by 420
A Sequential Method Seeking the Global Maximum of a Function
Shubert, B. O. (1972) · SIAM Journal on Numerical Analysis · doi:10.1137/0709036 · Cited by 407
Newton-Type Minimization via the Lanczos Method
Nash, S. G. (1984) · SIAM Journal on Numerical Analysis · doi:10.1137/0721052 · Cited by 338
Newton-type methods for unconstrained and linearly constrained optimization
Gill, P. E., & Murray, W. (1974) · Mathematical Programming · doi:10.1007/bf01585529 · Cited by 335
Global Convergence of a Cass of Quasi-Newton Methods on Convex Problems
Byrd, R. H., Nocedal, J., & Yuan, Y.-X. (1987) · SIAM Journal on Numerical Analysis · doi:10.1137/0724077 · Cited by 319
Global Convergence of a Class of Trust Region Algorithms for Optimization with Simple Bounds
Conn, A. R., Gould, N. I. M., & Toint, P. L. (1988) · SIAM Journal on Numerical Analysis · doi:10.1137/0725029 · Cited by 317
Computing Optimal Locally Constrained Steps
Gay, D. M. (1981) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0902016 · Cited by 293
Goldstein, A. A. (1967) · Defense Technical Information Center (DTIC) · Cited by 267
Morè, J. J., & Wright, S. J. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5498 · Cited by 204
A New Modified Cholesky Factorization
Schnabel, R. B., & Eskow, E. (1990) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0911064 · Cited by 163
Optimization on Microcomputers: The Nelder-Mead Simplex Algorithm
Dennis, J. E., E., W. J., & Daniel, J. (1985) · doi:10.21236/ada453814 · Cited by 97
Parallel quasi-Newton methods for unconstrained optimization
Byrd, R. H., Schnabel, R. B., & Shultz, G. A. (1988) · Mathematical Programming · doi:10.1007/bf01589407 · Cited by 73
Concurrent stochastic methods for global optimization
Byrd, R. H., Dert, C. L., Kan, A. H. G. R., & Schnabel, R. B. (1990) · Mathematical Programming · doi:10.1007/bf01585724 · Cited by 73
On Large Scale Nonlinear Least Squares Calculations
Toint, P. L. (1987) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0908042 · Cited by 45
A Generalized Broyden’s Method for Solving Simultaneous Linear Equations
Gerber, R. R., & Luk, F. T. (1981) · SIAM Journal on Numerical Analysis · doi:10.1137/0718061 · Cited by 22
Cited by
Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods
Kolda, T. G., Lewis, R. M., & Torczon, V. (2003) · SIAM Review · doi:10.1137/s003614450242889 · Cited by 1,653
Recent approaches to global optimization problems through Particle Swarm Optimization
Parsopoulos, K. E., & Vrahatis, M. N. (2002) · Natural Computing · doi:10.1023/a:1016568309421 · Cited by 1,406
Derivative-free optimization: a review of algorithms and comparison of software implementations
Rios, L. M., & Sahinidis, N. V. (2013) · Journal of Global Optimization · doi:10.1007/s10898-012-9951-y · Cited by 1,276
Implementing the Nelder-Mead simplex algorithm with adaptive parameters
Gao, F., & Han, L. (2012) · Computational Optimization and Applications · doi:10.1007/s10589-010-9329-3 · Cited by 1,090
Particle Swarm Optimization for Single Objective Continuous Space Problems: A Review
Bonyadi, M. R., & Michalewicz, Z. (2017) · Evolutionary Computation · doi:10.1162/evco_r_00180 · Cited by 841
Metadata
- Authors
- Kelley, C. T.
- Venue
- Society for Industrial and Applied Mathematics eBooks
- Published
- 1999-01
- OpenAlex
- W2123224804
- Cited by
- 2,052
- References
- 67
- FWCI
- 3.18
- Type
- book
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.