The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
The Tradeoffs of Large-Scale Learning
Abstract
This contribution develops a theoretical framework that takes into account the effect of approximate optimization on learning algorithms. The analysis shows distinct tradeoffs for the case of small-scale and large-scale learning problems. Small-scale learning problems are subject to the usual approximation–estimation tradeoff. Largescale learning problems are subject to a qualitatively different tradeoff involving the computational complexity of the underlying optimization algorithm in non-trivial ways. For instance, a mediocre optimization algorithms, stochastic gradient descent, is shown to perform very well on large-scale learning problems. 1
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895
Numerical Methods for Unconstrained Optimization and Nonlinear Equations
Dennis, J. E., & Schnabel, R. B. (1996) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611971200 · Cited by 7,615
Pattern Classification and Scene Analysis.
Clarke, M. R. B., Duda, R. O., & Hart, P. E. (1974) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344977 · Cited by 3,222
RCV1: A New Benchmark Collection for Text Categorization Research
Lewis, D., Yang, Y., Rose, T. G., & Li, F. (2004) · Goldsmiths (University of London) · doi:10.5555/1005332.1005345 · Cited by 2,503
Estimation of Dependences Based on Empirical Data
Vapnik, V. (2006) · Information Science and Statistics · doi:10.1007/0-387-34239-7 · Cited by 2,238
Training linear SVMs in linear time
Joachims, T. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150429 · Cited by 1,920
Shallow parsing with conditional random fields
Sha, F., & Pereira, F. (2003) · Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL '03 · doi:10.3115/1073445.1073473 · Cited by 1,246
Convexity, Classification, and Risk Bounds
Bartlett, P. L., Jordan, M. I., & McAuliffe, J. D. (2006) · Journal of the American Statistical Association · doi:10.1198/016214505000000907 · Cited by 1,037
Pegasos: Primal Estimated sub-GrAdient SOlver for SVM
Shalev-Shwartz, S., Singer, Y., & Srebro, N. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273598 · Cited by 973
Estimation of Dependences Based on Empirical Data.
Shoesmith, E., Vapnik, V., & Kotz, S. (1984) · The Statistician · doi:10.2307/2988246 · Cited by 764
Statistical behavior and consistency of classification methods based on convex risk minimization
Zhang, T. (2004) · The Annals of Statistics · doi:10.1214/aos/1079120130 · Cited by 711
Introduction to the CoNLL-2000 shared task: chunking
Tjong Kim Sang, E. F., & Buchholz, S. (2000) · Proceedings of the 2nd workshop on Learning language in logic and the 4th conference on Computational natural language learning - · doi:10.3115/1117601.1117631 · Cited by 705
Optimal aggregation of classifiers in statistical learning
Tsybakov, A. B. (2004) · The Annals of Statistics · doi:10.1214/aos/1079120131 · Cited by 582
Theory of Classification: a Survey of Some Recent Advances
Boucheron, S., Bousquet, O., & Lugosi, G. (2005) · ESAIM: Probability and Statistics · doi:10.1051/ps:2005018 · Cited by 436
Self-citationMeasuring the VC-Dimension of a Learning Machine
Vapnik, V., Levin, E., & Cun, Y. L. (1994) · Neural Computation · doi:10.1162/neco.1994.6.5.851 · Cited by 388
Bottou, L., & Cun, Y. L. (2003) · Cited by 359
Self-citationVapnik, V. N. (1982) · Springer eBooks · Cited by 295
Trust region Newton methods for large-scale logistic regression
Lin, C.-J., Weng, R. C., & Keerthi, S. S. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273567 · Cited by 284
Accelerated training of conditional random fields with stochastic gradient methods
Vishwanathan, S. V. N., Schraudolph, N. N., Schmidt, M. W., & Murphy, K. P. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143966 · Cited by 278
SVM optimization: inverse dependence on training set size
Shalev-Shwartz, S., & Srebro, N. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390273 · Cited by 240
Some applications of concentration inequalities to statistics
Massart, P. (2000) · Annales de la Faculté des sciences de Toulouse : Mathématiques · doi:10.5802/afst.961 · Cited by 240
On the complexity of loading shallow neural networks
Judd, S. (1988) · Journal of Complexity · doi:10.1016/0885-064x(88)90019-2 · Cited by 186
Bartlett, P. L., & Mendelson, S. (2006) · Probability Theory and Related Fields · doi:10.1007/s00440-005-0462-3 · Cited by 176
A Few Notes on Statistical Learning Theory
Mendelson, S. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-36434-x_1 · Cited by 161
Bousquet, O. (2002) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 122
Self-citationThe importance of convexity in learning with squared loss
Lee, W. S., Bartlett, P. L., & Williamson, R. C. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.705577 · Cited by 103
A Statistical Study of On-line Learning
Murata, N. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9780511569920.005 · Cited by 93
Fast Rates for Support Vector Machines
Steinwart, I., & Scovel, C. (2005) · Lecture notes in computer science · doi:10.1007/11503415_19 · Cited by 88
QP Algorithms with Guaranteed Accuracy and Run Time for Support Vector Machines
Hush, D. R., Kelly, P. J., Scovel, C., & Steinwart, I. (2006) · doi:10.5555/1248547.1248573 · Cited by 73
Valiant, L. G. (2026) · Foundations of Computation and Machine Learning · doi:10.1145/3749740.3749746
Cited by
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
Self-citationGeneralizing from a Few Examples: A Survey on Few-shot Learning
Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2021) · ACM Computing Surveys · doi:10.1145/3386252 · Cited by 3,165
Optimization Methods for Large-Scale Machine Learning
Bottou, L., Curtis, F. E., & Nocedal, J. (2018) · SIAM Review · doi:10.1137/16m1080173 · Cited by 3,128
Self-citation(2015) · Cited by 2,857
DeepXDE: A Deep Learning Library for Solving Differential Equations
Lu, L., Meng, X., Mao, Z., & Karniadakis, G. E. (2021) · SIAM Review · doi:10.1137/19m1274067 · Cited by 2,512
Metadata
- Authors
- Bottou, L., & Bousquet, O.
- Venue
- Optimization for Machine Learning
- Publisher
- The MIT Press
- Published
- 2011-09-30
- Pages
- 351-368
- OpenAlex
- W2113651538
- Cited by
- 1,128
- References
- 31
- FWCI
- 214.40
- Type
- book chapter
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.