Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 5,201–5,240 of 5,240
Alotaibi, N., & BinSaeedan, W. M. (2026) · Drones · doi:10.3390/drones10100721
shen, G., li, L., & Zhao, E. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7513272
Task-Performance Routing for Multi-teacher Distillation in CT Medical Image Segmentation
Yang, J., Liu, Y., Dai, D., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38082-1_57
Anatomy-Aware Reverse Symmetric Self-paced Learning for Prostate Cancer Detection
Zhang, D., Zhao, D., Sun, Y., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38091-3_6
Tan, Z., Karaköse, E., & Özbey, N. (2026) · IEEE Access · doi:10.1109/access.2026.3734386
“To reduce sparse-reward learning instability, a progressive curriculum-learning strategy was developed based on curriculum-learning principles introduced in [8].”
Hu, Z., Peng, C., Peng, Z., Cai, Z., Zhang, Z., Sun, Z., Hu, J., & Shouyu, L. (2026) · Mathematics · doi:10.3390/math14193479
MDGR: Masked Diffusion Generative Recommendation
Mu, L., Deng, H., Xing, H., Hu, J., Zhang, Y., Zeng, X., & Zhang, J. (2026) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3773078.3831862 · Cited by 19
Fair Curriculum Learning for Concept Bottleneck Models in Dermatology
Cockayne, M. J., Ortolani, M., & Al‐Bander, B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38079-1_26
Soft Curriculum Learning for Optimizing Fresh and Generalized Recommendations
Bhadury, A., Zheng, S., Yu, A., et al. (2026) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3773078.3831904
“The foundational concept of Curriculum Learning (CL) was formalized by Bengio et al. [1].”
Graph Neural Network‐Based Reinforcement Learning for Decentralized Multi‐Robot Manipulation
Chen, T., Fu, B., Tilbury, D. M., et al. (2026) · Advanced Intelligent Systems · doi:10.1002/aisy.70562
Multimodal Survival Analysis on Hierarchical Representation of Clinical Trial Data
Das, J., Mamann, A., Bugiotti, F., et al. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-39823-9_11
Exploring Transformers for Predicting the Remaining Useful Life of Turbofan Engines
Cabral, E. C., & Inacio, E. C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39882-6_42
Alam, E., Littlefield, N. G., Martin, J. R., & Moradi, H. (2025) · Studies in computational intelligence · doi:10.1007/978-3-032-11999-5_7
Zhou, W., Liu, S., Zhou, Z., Duan, Y., 蔡永岩, Wu, W., Li, T., Zhang, D., & Yu, Z. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3832008 · Cited by 1
LabelRoute: Routing by Instance-Label Hardness for Multi-label Classification
Lopes, A. Q., Brandão, H., Faria, E. R., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39895-6_10
UniPET: a unified approach for whole-body CT to PET translation
Feola, F. D., Guarrasi, V., Johansson, M., & Soda, P. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1919564
Li, Q., Liu, H., Zhang, J., et al. (2026) · Remote Sensing · doi:10.3390/rs18193368
Origin–Destination Proportion Analysis Using Machine Learning on Non-Overlapping Camera Videos
Kumar, S., Chien, S., Chen, Y., et al. (2026) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr70517.2026.00062
“Three encoders are used in this study: 1) Curriculum-trained TransReID: This is a three-stage hard-mining curriculum encoder [26] that generates progressively harder same-class same-color triplets, biasing toward parts of the vehicle that survive partial occlusion, such as the windshield frame,…”
Confidence-driven lightweight adaptive online deep learning for IoT intrusion detection
Agbedanu, P. R., Musabe, R., & Gatare, I. (2026) · Discover Telecommunications · doi:10.1007/s44595-026-00001-9
“The approach used in this study is inspired by curriculum and self-paced learning [41, 42], which schedules learning based on prediction confidence.”
Chen, H., Wang, J., Yan, J., & Zhang, X. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.115003
Adaptation and Interpretation of Various Instance Hardness Assessments for Text Spans
Maysuradze, A. I., & Tabachenkov, A. (2026) · Lobachevskii Journal of Mathematics · doi:10.1134/s1995080226618540
“…plays a pivotal role in estimating the object importance [3], data pruning [4], over-fi tting prevention and regularization [5], visualization [6], anomaly, out-of-domain and mislabeling detection [1, 2], curriculum learning [7], data re-annotation [1, 2], active learning and crowd-sourcing [2].”
Himantha, C., & Peiris, S. M. (2026) · doi:10.1109/wsai70435.2026.11708582
“Curriculum learning was introduced by Bengio et al. [8], who showed that ordering training examples from easy to hard improves both convergence speed and final accuracy.”
Curriculum-Aware Advantage Weighting and Coordination Regularization for Multi-Agent RL
Zhong, H., Jin, Y., & Li, X. (2026) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case69030.2026.11704352
“CR [19] has been used in RL to gradually increase task complexity, improving convergence and generalization.”
Dynamic Grid World Navigation using Q-Learning with Moving Obstacles and Terrain-Based Rewards
Babu, C., Hegde, A. S., Rao, S. H., et al. (2026) · IEEE World Conference on Applied Intelligence and Computing (AIC) · doi:10.1109/aic70896.2026.11717241
“The learning can further improve convergence by training the agent progressively from simpler to more complex environments [16].”
Curriculum-Trained Vertical Agents for Italian Medical Question Answering
Acunzo, A., Barone, M., Dioguardi, G., Marino, R. D., Serio, F. D., Iozzino, G., Postiglione, M., Riccio, G. E., Romano, A., & Vitiello, L. (2026) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3794786.3830763
“Rather than presenting training examples in random order, we adopt Curriculum Learning (CL) [10] to impose a principled difficulty ordering, promoting more stable optimization and better generalization.”
Kothwar, L., & Sahare, P. (2026) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-026-00622-5
Disentangled Speaker Flow with Time-Decayed Conditioning for Dialogue Text-to-Speech
Meng, G. (2026) · International Conference on Computer Science and Blockchain (CCSB) · doi:10.1109/ccsb71176.2026.11716520
“To ensure stable speech-text alignment in complex dialogues, it employs a curriculum learning [30] strategy, fine-tuning the pre-trained ZipVoice model on conversational dataset Opendialog.”
Simulation-Driven LiDAR Reinforcement Learning for Obstacle Jumping on the Unitree GO2
Rheman, T., & Somfai, E. (2026) · Intelligent Robotics FAIR · doi:10.1145/3831600.3831624
Emergent Co-Adaptive Strategies in Heterogeneous Multi-Robot Systems via Meta-Learning
Wang, H., Wang, L., Lam, T. L., Zhai, J., Lin, D., Zheng, H., He, X., & Gao, Y. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11697439
“…[27], enhanced by three modules: (i) scaled dot-product attention for efficient observation aggregation in large-scale environments [28], (ii) an Intrinsic Curiosity Module (ICM) [29] to provide exploration incentives under sparse rewards, and (iii) curriculum learning [30] to stabilize training.”
Targeted Behavioral Unlearning for Discrete On-Policy Reinforcement Learning
Takács, T., & Gulyás, L. (2026) · Intelligent Robotics FAIR · doi:10.1145/3831600.3831610
Zhao, X., & Fang, Z. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11696427
Harnessing Uncertainty in Code Language Models: Lessons from Vulnerability Detection
Li, H., Cheng, X. Y., Wang, X., Guo, Z., & Wang, H. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3837465
CASTNet: a content-adaptive spatio-temporal network for robust video hiding
Lu, Y., Niu, K., & Zhu, S. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01322-0
“To address this issue, we adopt a Progressive Training Strategy (PTS) that gradually injects compression perturbations into the training process from weak to strong (Bengio et al. 2009).”
Using Instance-Level Meta-Information to Facilitate a More Principled Approach to Machine Learning
Smith, M. R. (2015) · ScholarsArchive (Brigham Young University)
Layered Programming by Demonstration and Planning for Autonomous Robot Manipulation
Jäkel, R., Ruehl, S. W., Schmidt-Rohr, S. R., Lösch, M., Xue, Z., & Dillmann, R. (2012) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-29041-1_1
Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory
Suzgun, M., Yuksekgonul, M., Bianchi, F., Jurafsky, D., & Zou, J. (2025) · arXiv · doi:10.48550/arxiv.2504.07952
Reflect-RL: Two-Player Online RL Fine-Tuning for LMs
Zhou, R., Du, S. S., & Li, B. (2024) · arXiv · doi:10.48550/arxiv.2402.12621
Dai, R., Jian, A., Zhang, R., Jia, Y., Mo, L., Zhan, S., & He, T. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116272
SLR: Automated Synthesis for Scalable Logical Reasoning
Helff, L., Omar, A., Friedrich, F., Wüst, A., Shindo, H., Mitchell, R., Woydt, T., Schramowski, P., Stammer, W., & Kersting, K. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.16
Learning Compositional Functions with Transformers from Easy-to-Hard Data
Wang, Z., Nichani, E., Bietti, A., Damian, A., Hsu, D., Lee, J. D., & Wu, D. (2025) · arXiv · doi:10.48550/arxiv.2505.23683
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.