The DOI matches its Crossref record; 97 of the 97 references it cites were found in a registry or an index.
Optimization Algorithms on Matrix Manifolds
Abstract
Many problems in the sciences and engineering can be rephrased as optimization problems on matrix search spaces endowed with a so-called manifold structure. This book shows how to exploit the special structure of such problems to develop efficient numerical algorithms. It places careful emphasis on both the numerical formulation of the algorithm and its differential geometric abstraction--illustrating how good algorithms draw equally from the insights of differential geometry, optimization, and numerical analysis. Two more theoretical chapters provide readers with the background in differential geometry necessary to algorithmic development. In the other chapters, several well-known optimization methods such as steepest descent and conjugate gradients are generalized to abstract manifolds. The book provides a generic development of each of these methods, building upon the material of the geometric chapters. It then guides readers through the calculations that turn these geometrically formulated methods into concrete numerical algorithms. The state-of-the-art algorithms given as examples are competitive with the best existing algorithms for a selection of eigenspace problems in numerical linear algebra. Optimization Algorithms on Matrix Manifolds offers techniques with broad applications in linear algebra, signal processing, data mining, computer vision, and statistical analysis. It can serve as a graduate-level textbook and will be of interest to applied mathematicians, engineers, and computer scientists.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Algorithm for Least-Squares Estimation of Nonlinear Parameters
Marquardt, D. W. (1963) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0111030 · Cited by 30,728
Horn, R. A., & Johnson, C. R. (1985) · Cited by 22,203
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Iterative Methods for Sparse Linear Systems
Saad, Y. (2003) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898718003 · Cited by 13,790
Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 11,726
Applied Multivariate Statistical Analysis.
Johnson, A. R., & Wichern, D. W. (1988) · Biometrics · doi:10.2307/2531616 · Cited by 11,353
Horn, R. A., & Johnson, C. R. (1991) · Cambridge University Press eBooks · doi:10.1017/cbo9780511840371 · Cited by 9,319
Methods of conjugate gradients for solving linear systems
Hestenes, M. R., & Stiefel, E. (1952) · Journal of Research of the National Bureau of Standards · doi:10.6028/jres.049.044 · Cited by 8,132
Numerical Methods for Unconstrained Optimization and Nonlinear Equations
Dennis, J. E., & Schnabel, R. B. (1996) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611971200 · Cited by 7,615
Practical Methods of Optimization
Fletcher, R. (2000) · Wiley eBooks · doi:10.1002/9781118723203 · Cited by 7,605
Iterative Solution of Nonlinear Equations in Several Variables
Ortega, J. M., & Rheinboldt, W. C. (2000) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898719468 · Cited by 6,860
“Neural” computation of decisions in optimization problems
Hopfield, J. J., & Tank, D. W. (1985) · Biological Cybernetics · doi:10.1007/bf00339943 · Cited by 6,031
The algebraic eigenvalue problem
Wilkinson, J. H. (1965) · Cited by 5,833
Function minimization by conjugate gradients
Fletcher, R. (1964) · The Computer Journal · doi:10.1093/comjnl/7.2.149 · Cited by 4,939
The Approximation of One Matrix by Another of Lower Rank
Eckart, C., & Young, G. (1936) · Psychometrika · doi:10.1007/bf02288367 · Cited by 3,800
Differential geometry of curves and surfaces
(2011) · Choice Reviews Online · doi:10.5860/choice.48-3330 · Cited by 3,706
A Comprehensive Introduction to Differential Geometry.
Kramer, A. D., & Spivak, M. (1973) · The American Mathematical Monthly · doi:10.2307/2319112 · Cited by 3,445
Matrix Differential Calculus with Applications in Statistics and Econometrics.
Magnus, J. R., & Neudecker, H. (1988) · Biometrics · doi:10.2307/2531754 · Cited by 2,699
The Geometry of Algorithms with Orthogonality Constraints
Edelman, A., Arias, T. A., & Smith, S. T. (1998) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/s0895479895290954 · Cited by 2,685
Numerical solution of saddle point problems
Benzi, M., Golub, G. H., & Liesen, J. (2005) · Acta Numerica · doi:10.1017/s0962492904000212 · Cited by 2,300
Axelsson, O. (1994) · Cambridge University Press eBooks · doi:10.1017/cbo9780511624100 · Cited by 2,085
Minimization of functions having Lipschitz continuous first partial derivatives
Armijo, L. (1966) · Pacific Journal of Mathematics · doi:10.2140/pjm.1966.16.1 · Cited by 1,983
Introduction to linear and nonlinear programming
Luenberger, D. G. (1973) · Medical Entomology and Zoology · Cited by 1,915
Foundations of Differentiable Manifolds and Lie Groups
Warner, F. W. (1983) · Graduate Texts in Mathematics · doi:10.1007/978-1-4757-1799-0 · Cited by 1,848
Computational Methods of Linear Algebra
H., A. S., Faddeev, D. K., & Faddeeva, V. N. (1964) · Mathematics of Computation · doi:10.2307/2003785 · Cited by 1,599
Moré, J. J., & Sorensen, D. C. (1983) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0904038 · Cited by 1,411
Projection approximation subspace tracking
Yang, B. (1995) · IEEE Transactions on Signal Processing · doi:10.1109/78.365290 · Cited by 1,273
On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations I
Fan, K. (1949) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.35.11.652 · Cited by 1,016
Note sur la convergence de méthodes de directions conjuguées
Polak, E., & Ribiere, G. (1969) · Revue française d'informatique et de recherche opérationnelle. Série rouge · doi:10.1051/m2an/196903r100351 · Cited by 986
Computational methods in optimization
Polak, E. (1971) · Cited by 941
A Jacobi–Davidson Iteration Method for Linear Eigenvalue Problems
G. Sleijpen, G. L., & Van der Vorst, H. A. (1996) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/s0895479894270427 · Cited by 848
The Conjugate Gradient Method and Trust Regions in Large Scale Optimization
Steihaug, T. (1983) · SIAM Journal on Numerical Analysis · doi:10.1137/0720042 · Cited by 824
NEURAL NETWORKS, PRINCIPAL COMPONENTS, AND SUBSPACES
Oja, E. (1989) · International Journal of Neural Systems · doi:10.1142/s0129065789000475 · Cited by 821
Knyazev, A. V. (2001) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827500366124 · Cited by 778
Linear and nonlinear programming
(2005) · Economists’ Mathematical Manual · doi:10.1007/3-540-28518-0_15 · Cited by 766
Optimization and Dynamical Systems
Helmke, U., & Moore, J. B. (1994) · Communications and Control Engineering · doi:10.1007/978-1-4471-3467-1 · Cited by 724
Matrices, Vector Spaces, and Information Retrieval
Berry, M. W., Drmac, Z., & Jessup, E. R. (1999) · SIAM Review · doi:10.1137/s0036144598347035 · Cited by 692
A D.C. Optimization Algorithm for Solving the Trust-Region Subproblem
Dinh, T. P., & Thi, H. A. L. (1998) · SIAM Journal on Optimization · doi:10.1137/s1052623494274313 · Cited by 569
Independent component analysis for biomedical signals
James, C. J., & Hesse, C. W. (2005) · Physiological Measurement · doi:10.1088/0967-3334/26/1/r02 · Cited by 494
Tracking a few extreme singular values and vectors in signal processing
Comon, P., & Golub, G. H. (1990) · Proceedings of the IEEE · doi:10.1109/5.58320 · Cited by 489
Adaptive blind signal processing-neural network approaches
Amari, S., & Cichocki, A. (1998) · Proceedings of the IEEE · doi:10.1109/5.720251 · Cited by 466
Trust-Region Methods on Riemannian Manifolds
Absil, P.-A., Baker, C. G., & Gallivan, K. A. (2007) · Foundations of Computational Mathematics · doi:10.1007/s10208-005-0179-9 · Cited by 461
Self-citationOptimization algorithms exploiting unitary constraints
Manton, J. H. (2002) · IEEE Transactions on Signal Processing · doi:10.1109/78.984753 · Cited by 457
A New Algorithm for Unconstrained Optimization
POWELL, M. J. D. (1970) · Nonlinear Programming · doi:10.1016/b978-0-12-597050-1.50006-3 · Cited by 447
Approximate solution of the trust region problem by minimization over two-dimensional subspaces
Byrd, R. H., Schnabel, R. B., & Shultz, G. A. (1988) · Mathematical Programming · doi:10.1007/bf01580735 · Cited by 411
Functional analysis and applied mathematics
Kantorovich, L. V., & Benster, C. D. (1963) · University Microfilms International eBooks · Cited by 410
Symplectic integrators for Hamiltonian problems: an overview
Sanz-Serna, J. M. (1992) · Acta Numerica · doi:10.1017/s0962492900002282 · Cited by 400
Yeredor, A. (2002) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2002.1011195 · Cited by 397
Riemannian Geometry of Grassmann Manifolds with a View on Algorithmic Computation
Absil, P.-A., Mahony, R., & Sepulchre, R. (2004) · Acta Applicandae Mathematica · doi:10.1023/b:acap.0000013855.14971.91 · Cited by 393
Self-citationEigenvalue computation in the 20th century
Golub, G. H., & van der Vorst, H. A. (2000) · Journal of Computational and Applied Mathematics · doi:10.1016/s0377-0427(00)00413-1 · Cited by 388
Convergence of the Iterates of Descent Methods for Analytic Cost Functions
Absil, P. A., Mahony, R., & Andrews, B. (2005) · SIAM Journal on Optimization · doi:10.1137/040605266 · Cited by 291
Self-citationSolving the Trust-Region Subproblem using the Lanczos Method
Gould, N. I. M., Lucidi, S., Roma, M., & Toint, P. L. (1999) · SIAM Journal on Optimization · doi:10.1137/s1052623497322735 · Cited by 280
Newton's method on Riemannian manifolds and a geometric model for the human spine
Adler, R. L. (2002) · IMA Journal of Numerical Analysis · doi:10.1093/imanum/22.3.359 · Cited by 271
Jacobi-davidson type methods for generalized eigenproblems and polynomial eigenproblems
Sleijpen, G. L. G., Booten, A. G. L., Fokkema, D. R., & van der Vorst, H. A. (1996) · BIT Numerical Mathematics · doi:10.1007/bf01731936 · Cited by 267
Generalizations of Davidson’s Method for Computing Eigenvalues of Sparse Symmetric Matrices
Morgan, R. B., & Scott, D. S. (1986) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0907054 · Cited by 186
Inverse Iteration, Ill-Conditioned Equations and Newton’s Method
Peters, G., & Wilkinson, J. H. (1979) · SIAM Review · doi:10.1137/1021052 · Cited by 183
Numerical methods for large eigenvalue problems
Sorensen, D. C. (2002) · Acta Numerica · doi:10.1017/s0962492902000089 · Cited by 162
Wang, J., & Liang, X.-B. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.883412 · Cited by 160
Fast subspace tracking and neural network learning by a novel information criterion
Miao, Y., & Hua, Y. (1998) · IEEE Transactions on Signal Processing · doi:10.1109/78.700968 · Cited by 142
Optimization Criteria and Geometric Algorithms for Motion and Structure Estimation
Ma, Y., Košecká, J., & Sastry, S. (2001) · International Journal of Computer Vision · doi:10.1023/a:1012276232049 · Cited by 141
A Procrustes problem on the Stiefel manifold
Eldén, L., & Park, H. (1999) · Numerische Mathematik · doi:10.1007/s002110050432 · Cited by 129
Least-Squares Covariance Matrix Adjustment
Boyd, S., & Xiao, L. (2005) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/040609902 · Cited by 126
Newton's method on Riemannian manifolds: covariant alpha theory
Dedieu, J.-P. (2003) · IMA Journal of Numerical Analysis · doi:10.1093/imanum/23.3.395 · Cited by 120
Optimal linear representations of images for object recognition
Liu, X., Srivastava, A., & Gallivan, K. A. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1273986 · Cited by 115
Bayesian and geometric subspace tracking
Srivastava, A., & Klassen, E. (2004) · Advances in Applied Probability · doi:10.1017/s0001867800012866 · Cited by 111
Bi-iteration SVD subspace tracking algorithms
Strobach, P. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.575696 · Cited by 111
Combination of Jacobi–Davidson and conjugate gradients for the partial symmetric eigenproblem
Notay, Y. (2001) · Numerical Linear Algebra with Applications · doi:10.1002/nla.246 · Cited by 106
Joho, M., & Mathis, H. (2003) · IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM) · doi:10.1109/sam.2002.1191043 · Cited by 100
The trace minimization method for the symmetric generalized eigenvalue problem
Sameh, A., & Tong, Z. (2000) · Journal of Computational and Applied Mathematics · doi:10.1016/s0377-0427(00)00391-5 · Cited by 99
Nonholonomic Orthogonal Learning Algorithms for Blind Source Separation
Амари, Ш.-И., Chen, T., & Cichocki, A. S. (2000) · Neural Computation · doi:10.1162/089976600300015466 · Cited by 99
A New Look at the Power Method for Fast Subspace Tracking
Hua, Y., Xiang, Y., Chen, T., Abed‐Meraim, K., & Miao, Y. (1999) · Digital Signal Processing · doi:10.1006/dspr.1999.0348 · Cited by 94
Methods for the approximation of the matrix exponential in a Lie-algebraic setting
Celledoni, E. (2001) · IMA Journal of Numerical Analysis · doi:10.1093/imanum/21.2.463 · Cited by 85
Unified stabilization approach to principal and minor components extraction algorithms
Chen, T., & Амари, Ш.-И. (2001) · Neural Networks · doi:10.1016/s0893-6080(01)00116-2 · Cited by 83
Numerical Linear Algebra Techniques for Systems and Control
Patel, R. V., Laub, A. J., & Dooren, P. M. V. (1993) · Cited by 80
Geometrical methods for non-negative ICA: Manifolds, Lie groups and toral subalgebras
Plumbley, M. D. (2005) · Neurocomputing · doi:10.1016/j.neucom.2004.11.040 · Cited by 79
Stathopoulos, A. (2007) · SIAM Journal on Scientific Computing · doi:10.1137/050631574 · Cited by 75
The Geometry of the Newton Method on Non-Compact Lie Groups
Mahony, R. E., & Manton, J. H. (2002) · Journal of Global Optimization · doi:10.1023/a:1016586831090 · Cited by 70
Self-citationEssential Matrix Estimation Using Gauss-Newton Iterations on a Manifold
Uwe Helmke, B., Hüper, K., Lee, P. Y., & Moore, J. B. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-0005-0 · Cited by 70
Self-stabilized gradient algorithms for blind source separation with orthogonality constraints
Douglas, S. C. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.883482 · Cited by 69
Sleijpen, G. L. G., van der Vorst, H. A., & Meijerink, E. (1998) · Cited by 67
A Bayesian approach to geometric subspace estimation
Srivastava, A. (2000) · IEEE Transactions on Signal Processing · doi:10.1109/78.839985 · Cited by 60
Minimum-seeking properties of analog neural networks with multilinear objective functions
Vidyasagar, M. (1995) · IEEE Transactions on Automatic Control · doi:10.1109/9.402228 · Cited by 59
Numerical Gradient Algorithms for Eigenvalue and Singular Value Calculations
Moore, J. B., Mahony, R. E., & Helmke, U. (1994) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/s0036141092229732 · Cited by 56
Self-citationSequential Subspace Optimization Method for Large-Scale Unconstrained Problems
Narkiss, G., & Zibulevsky, M. (2005) · Cited by 55
Convergence Analysis of Inexact Rayleigh Quotient Iteration
Notay, Y. (2003) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/s0895479801399596 · Cited by 53
A truncated-CG style method for symmetric generalized eigenvalue problems
Absil, P.-A., Baker, C. G., & Gallivan, K. A. (2005) · Journal of Computational and Applied Mathematics · doi:10.1016/j.cam.2005.10.006 · Cited by 47
Self-citationDifferentiable families of subspaces
Ferrer, J., García, M. I., & Puerta, F. (1994) · Linear Algebra and its Applications · doi:10.1016/0024-3795(94)90351-4 · Cited by 44
Restarting techniques for the (Jacobi-)Davidson symmetric eigenvalue methods
Stathopoulos, A., & Saad, Y. E.-M. (1998) · Cited by 44
The Riccati algorithm for eigenvalues and invariant subspaces of matrices with inexpensive action
Brandts, J. (2002) · Linear Algebra and its Applications · doi:10.1016/s0024-3795(02)00392-0 · Cited by 31
Zhou, Y. (2006) · Numerical Linear Algebra with Applications · doi:10.1002/nla.490 · Cited by 31
A dual purpose principal and minor component flow
Manton, J. H., Helmke, U., & Mareels, I. (2005) · Systems & Control Letters · doi:10.1016/j.sysconle.2004.11.015 · Cited by 28
A modified block Newton iteration for approximating an invariant subspace of a symmetric matrix
Lösche, R., Schwetlick, H., & Timmermann, G. (1998) · Linear Algebra and its Applications · doi:10.1016/s0024-3795(97)10018-0 · Cited by 25
Is Jacobi--Davidson Faster than Davidson?
Notay, Y. (2004) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/s0895479803430941 · Cited by 21
Balanced Realizations for Linear Systems: A Variational Approach
Helmke, U. (1993) · SIAM Journal on Control and Optimization · doi:10.1137/0331001 · Cited by 20
Gradient algorithms for principal component analysis
Mahony, R. E., Helmke, U., & Moore, J. B. (1996) · ANZIAM Journal · doi:10.1017/s033427000001078x · Cited by 17
Self-citationContinuous dynamical systems that realize discrete optimization on the hypercube
Absil, P.-A., & Sepulchre, R. J. (2004) · Systems & Control Letters · doi:10.1016/j.sysconle.2004.02.008 · Cited by 14
Self-citationA block Rayleigh quotient iteration with local quadratic convergence.
Fattebert, J. (1998) · Cited by 11
Cited by
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update
Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., & Yger, F. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aab2f2 · Cited by 2,239
Riemann manifold langevin and hamiltonian monte carlo methods
Girolami, M., & Calderhead, B. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00765.x · Cited by 1,473
Alternating Minimization Algorithms for Hybrid Precoding in Millimeter Wave MIMO Systems
Yu, X., Shen, J.-C., Zhang, J., & Letaief, K. B. (2016) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2016.2523903 · Cited by 1,444
Attouch, H., Bolte, J., Redont, P., & Soubeyran, A. (2010) · Mathematics of Operations Research · doi:10.1287/moor.1100.0449 · Cited by 1,172
Metadata
- Authors
- Absil, P.-A., Mahony, R., & Sepulchre, R.
- Venue
- Princeton University Press eBooks
- Publisher
- Princeton University Press
- Published
- 2008-12-31
- OpenAlex
- W1804110266
- Cited by
- 1,840
- References
- 97
- FWCI
- 5.21
- Type
- book
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.