Local Learning Algorithms
Abstract
Very rarely are training data evenly distributed in the input space. Local learning algorithms attempt to locally adjust the capacity of the training system to the properties of the training set in each area of the input space. The family of local learning algorithms contains known methods, like the k-nearest neighbors method (kNN) or the radial basis function networks (RBF), as well as new algorithms. A single analysis models some aspects of these algorithms. In particular, it suggests that neither kNN or RBF, nor nonlocal classifiers, achieve the best compromise between locality and capacity. A careful control of these parameters in a simple local learning algorithm has provided a performance breakthrough for an optical character recognition problem. Both the error rate and the rejection performance have been significantly improved.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cited by
Sorted by citations
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Self-citationMachine learning with big data: challenges and approaches.
L'Heureux, A., Grolinger, K., Elyamany, H. F., & Capretz, M. A. M. (2017) · IEEE Access · doi:10.1109/access.2017.2696365 · Cited by 1,023 · see citing works
Titans: Learning to Memorize at Test Time
Behrouz, A., Zhong, P., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2501.00663 · Cited by 24 · see citing works
“The key ideas of learning at test time or learning to learn (i.e., [115]) backs to very early studies on local learning [116], in which each test data sample is trained on its neighbors before making a prediction [117, 22].”
Learning to (Learn at Test Time): RNNs with Expressive Hidden States
Sun, Y., Li, X., Dalal, K., Xu, J., Vikram, A., Zhang, G., Dubois, Y., Chen, X., Wang, X., Koyejo, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.04620 · Cited by 18 · see citing works
“One of the earliest versions of this idea is called local learning (Bottou and Vapnik [9]): For each test input, train on its neighbors before making a prediction.”
TT-SI: Self-Improving LLM Agents with Test-Time Training
Acikgoz, E. C., Qian, C., Ji, H., Hakkani-Tür, D., & Tur, G. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.462 · Cited by 1 · see citing works
“The idea traces to local and trans-ductive learning, where hypotheses are adapted after observing test inputs (Bottou and Vapnik, 1992; Joachims, 1999”
End-to-End Test-Time Training for Long Context
Tandon, A., Dalal, K., Li, X., Koceja, D., Rød, M., Buchanan, S., Wang, X., Leskovec, J., Koyejo, S., Hashimoto, T., et al. (2025) · arXiv
“One simple form of test-time training was called locally weighted regression in the 1970s [85, 18], local learning in the 1990s [12], and KNN-SVM in the 2000s [109]: Given a test instance, find its nearest neighbors in the training set, and then train (or fine-tune) the model on these neighbors…”
Behrouz, A., Razaviyayn, M., Zhong, P., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2504.13173
Self-Improving LLM Agents at Test-Time
Acikgoz, E. C., Qian, C., Ji, H., Hakkani-Tür, D., & Tur, G. (2025) · arXiv
“Motivated by local and transductive learning (Bottou & Vapnik, 1992; Joachims, 1999) with recent advances in test-time fine-tuning (Akyürek et al., 2025), we investigate a simple, yet powerful, instance-specific self-improvement algorithm that adapts agents on-the-fly to each downstream task at…”
- Authors
- Bottou, L., & Vapnik, V.
- Affiliations
- AT&T (United States) (US)
- ORCID
- Bottou, Léon
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 1992-11
- Volume
- 4
- Issue
- 6
- Pages
- 888-900
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2056763477
- Cited by
- 542
- References
- 0
- Citations by year
- 2026: 8 · 2025: 11 · 2024: 15 · 2023: 18 · 2022: 14 · 2021: 15 · 2020: 23 · 2019: 24 · 2018: 24 · 2017: 24 · 2016: 17 · 2015: 32 · 2014: 17 · 2013: 27 · 2012: 19
- FWCI
- 8.89
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and ELM
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- k-nearest neighbors, radial basis function neural network, optical character recognition, error rate
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
