The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
A Convergent Incremental Gradient Method with a Constant Step Size
Abstract
An incremental aggregated gradient method for minimizing a sum of continuously differentiable functions is presented. The method requires a single gradient evaluation per iteration and uses a constant step size. For the case that the gradient is bounded and Lipschitz continuous, we show that the method visits infinitely often regions in which the gradient is small. Under certain unimodality assumptions, global convergence is established. In the quadratic case, a global linear rate of convergence is shown. The method is applied to distributed optimization problems arising in wireless sensor networks, and numerical experiments compare the new method with other incremental gradient methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884
Advanced engineering mathematics
(1998) · Choice Reviews Online · doi:10.5860/choice.35-5129 · Cited by 4,100
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,515
A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187
(1989) · Choice Reviews Online · doi:10.5860/choice.26-5109 · Cited by 1,755
Varberg, D. E. (1973) · Cited by 1,700
Distributed optimization in sensor networks
Rabbat, M., & Nowak, R. (2004) · Proceedings of the 3rd international symposium on Information processing in sensor networks · doi:10.1145/984622.984626 · Cited by 901
Sheng, X., & Hu, Y.-H. (2005) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2004.838930 · Cited by 714
Habitat monitoring with sensor networks
Szewczyk, R., Osterweil, E., Polastre, J., Hamilton, M., Mainwaring, A., & Estrin, D. (2004) · Communications of the ACM · doi:10.1145/990680.990704 · Cited by 702
Logistic Regression, AdaBoost and Bregman Distances
Collins, M., Schapire, R. E., & Singer, Y. (2002) · Machine Learning · doi:10.1023/a:1013912006537 · Cited by 687
Incremental Subgradient Methods for Nondifferentiable Optimization
Nedic, A., & Bertsekas, D. P. (2001) · SIAM Journal on Optimization · doi:10.1137/s1052623499362111 · Cited by 589
Source localization and beamforming
Chen, J. C., Yao, K., & Hudson, R. E. (2002) · IEEE Signal Processing Magazine · doi:10.1109/79.985676 · Cited by 553
Gradient Convergence in Gradient methods with Errors
Bertsekas, D. P., & Tsitsiklis, J. N. (2000) · SIAM Journal on Optimization · doi:10.1137/s1052623497331063 · Cited by 437
Quantized incremental algorithms for distributed optimization
Rabbat, M. G., & Nowak, R. D. (2005) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2005.843546 · Cited by 408
A New Class of Incremental Gradient Methods for Least Squares Problems
Bertsekas, D. P. (1997) · SIAM Journal on Optimization · doi:10.1137/s1052623495287022 · Cited by 312
Distributed EM algorithms for density estimation and clustering in sensor networks
Nowak, R. D. (2003) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2003.814623 · Cited by 256
Convergence of Approximate and Incremental Subgradient Methods for Convex Optimization
Kiwiel, K. C. (2004) · SIAM Journal on Optimization · doi:10.1137/s1052623400376366 · Cited by 182
The Iterated Kalman Smoother as a Gauss–Newton Method
Bell, B. M. (1994) · SIAM Journal on Optimization · doi:10.1137/0804035 · Cited by 163
On the Convergence of the LMS Algorithm with Adaptive Learning Rate for Linear Feedforward Networks
Luo, Z.-Q. T. (1991) · Neural Computation · doi:10.1162/neco.1991.3.2.226 · Cited by 161
An Incremental Gradient(-Projection) Method with Momentum Term and Adaptive Stepsize Rule
Tseng, P. (1998) · SIAM Journal on Optimization · doi:10.1137/s1052623495294797 · Cited by 160
The Ordered Subsets Mirror Descent Optimization Method with Applications to Tomography
Ben‐Tal, A., Margalit, T., & Nemirovski, A. S. (2001) · SIAM Journal on Optimization · doi:10.1137/s1052623499354564 · Cited by 157
Incremental Least Squares Methods and the Extended Kalman Filter
Bertsekas, D. P. (1996) · SIAM Journal on Optimization · doi:10.1137/s1052623494268522 · Cited by 150
Energy Based Acoustic Source Localization
Sheng, X., & Hu, Y. H. (2003) · Lecture notes in computer science · doi:10.1007/3-540-36978-3_19 · Cited by 146
Decentralized source localization and tracking
Rabbat, M., & Nowak, R. D. (2004) · Minds at UW (University of Wisconsin) · Cited by 142
Mathematical Programming in Neural Networks
Mangasarian, O. L. (1993) · INFORMS journal on computing · doi:10.1287/ijoc.5.4.349 · Cited by 105
Gaivoronski, A. A. (1994) · Optimization methods & software · doi:10.1080/10556789408805582 · Cited by 78
Convergent incremental optimization transfer algorithms: application to tomography
Ahn, S., Fessler, J. A., Blatt, D., & Hero, A. O. (2006) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2005.862740 · Cited by 71
Self-citationConvergent on-line algorithms for supervised learning in neural networks
Grippo, L. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.883426 · Cited by 45
Davidon, W. C. (1976) · Journal of Optimization Theory and Applications · doi:10.1007/bf00935703 · Cited by 42
Choosing parameters in block-iterative or ordered subset reconstruction algorithms
Byrne, C. L. (2005) · IEEE Transactions on Image Processing · doi:10.1109/tip.2004.841193 · Cited by 31
The Incremental Gauss-Newton Algorithm with Adaptive Stepsize Rule
Moriyama, H., Yamashita, N., & Fukushima, M. (2003) · Computational Optimization and Applications · doi:10.1023/a:1025703629626 · Cited by 30
Cited by
Optimization Methods for Large-Scale Machine Learning
Bottou, L., Curtis, F. E., & Nocedal, J. (2018) · SIAM Review · doi:10.1137/16m1080173 · Cited by 3,128
Constrained Consensus and Optimization in Multi-Agent Networks
Nedic, A., Ozdaglar, A., & Parrilo, P. A. (2010) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2010.2041686 · Cited by 2,086
Minimizing finite sums with the stochastic average gradient
Schmidt, M., Le Roux, N., & Bach, F. (2017) · Mathematical Programming · doi:10.1007/s10107-016-1030-6 · Cited by 922
Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization
Sundhar Ram, S., Nedić, A., & Veeravalli, V. V. (2010) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-010-9737-7 · Cited by 814
Fast Distributed Gradient Methods
Jakovetic, D., Xavier, J., & Moura, J. M. F. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2298712 · Cited by 624
Metadata
- Authors
- Blatt, D., Hero, A. O., & Gauchman, H.
- Venue
- SIAM Journal on Optimization
- Published
- 2007-01
- Volume
- 18
- Issue
- 1
- Pages
- 29-51
- ISSN
- 1052-6234 (ISSN-L)
- 1095-7189
- OpenAlex
- W1988795359
- Cited by
- 306
- References
- 31
- FWCI
- 7.88
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.