Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 701–800 of 5,240
Shortformer: Better language modeling using shorter inputs
Press, O., Smith, N. A., & Lewis, M. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.427 · Cited by 55
“All models are trained in two stages; the second stage always uses a subsequence length of 3 , 072 , 6 Curriculum learning (Bengio et al., 2009) trains on easier inputs before progressing to harder ones.”
BackgroundMethodOptimal Teaching for Limited-Capacity Human Learners
Patil, K. R., Zhu, X., Kopeć, Ł., & Love, B. C. (2014) · Cited by 56
Data Parameters: A New Family of Parameters for Learning a Differentiable Curriculum
Saxena, S., Tuzel, O., & DeCoste, D. (2019) · neural information processing systems · Cited by 59
Adaptive Sample Selection for Robust Learning under Label Noise
Patel, D., & SASTRY, P. S. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00392 · Cited by 92
“Curriculum learning (CL) as proposed in [4] is the designing of an (optimal) manner of sequencing of training samples (based on a notion of easiness of an example) to improve the performance of the learning system.”
BackgroundRobust Visual Question Answering: Datasets, Methods, and Future Challenges
Ma, J., Wang, P., Kong, D., Wang, Z., Liu, J., Pei, H., & Zhao, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3366154 · Cited by 87
“Curriculum learning [152] enables models to be trained initially on the easier examples and gradually shift towards the harder ones.”
BackgroundChen, K., Liu, S., Chen, B., Wang, H., Jan, E.-E., Hsu, W., & Chen, H. (2015) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2015.2432578 · Cited by 60
“The training strategy described above can also be viewed as an instantiation of curriculum learning [45], [46], which seeks to apply a specific and well-planned ordering of the training data for estimating machine-learning models (such as neural networks ) to be better suited for a target…”
BackgroundDecomposition-Based Evolutionary Multiobjective Optimization to Self-Paced Learning
Gong, M., Li, H., Meng, D., Miao, Q., & Liu, J. (2018) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2018.2850769 · Cited by 54
“to curriculum learning (CL) [2] proposed by Bengio et al.”
BackgroundSkin Lesion Classification in Dermoscopy Images Using Synergic Deep Learning
Zhang, J., Xie, Y., Wu, Q., & Xia, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00934-2_2 · Cited by 53
Structured Adversarial Training for Unsupervised Monocular Depth Estimation
Mehta, I., Sakurikar, P., & Narayanan, P. J. (2018) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00044 · Cited by 56
“[4], we argue that by training our framework with right views in a meaningful order helps the discriminator learn in a more structured manner.”
BackgroundWang, W., Caswell, I., & Chelba, C. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1123 · Cited by 46
“Curriculum learning (CL) (Bengio et al., 2009) has been used to further improve traditional static selection.”
MethodDance Revolution: Long-Term Dance Generation with Music via Curriculum LearningPreprint
Huang, R., Huang, H., Wu, W., Sawada, K. B., Zhang, M., & Jiang, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06119 · Cited by 175
“Furthermore, we propose a curriculum learning (Bengio et al., 2009) strategy to alleviate error accumulation (Li et al., 2017) of autoregressive models in long motion sequence generation.”
MethodA Survey on Transfer Learning in Natural Language ProcessingPreprint
Alyafeai, Z., Al-shaibani, M. S., & Ahmad, I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04239 · Cited by 134
“The later approach is called curriculum learning in the literature (Bengio et al., 2009).”
BackgroundWang, R., Lehman, J., Rawal, A., Zhi, J., Li, Y., Clune, J., & Stanley, K. O. (2020) · arXiv · doi:10.48550/arxiv.2003.08536 · Cited by 53
“Interestingly, much research effort is spent attempting to design or learn single-path curricula to help an agent learn a complex task (Bengio et al., 2009; Gomez & Miikkulainen, 1997; Heess et al., 2017; Justesen et al., 2018; Karpathy & Van De Panne, 2012).”
BackgroundHierarchical Query Graph Generation for Complex Question Answering over Knowledge Graph
Qiu, Y., Zhang, K., Wang, Y., Jin, X., Bai, L., Guan, S., & Cheng, X. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3411888 · Cited by 52
“Based on these high-reward trajectories, we leverage curriculum learning [6] to train the Director and the Actor gradually according to the complexity of questions during query graph generation.”
BackgroundMethodIntrinsic Evaluation of Summarization Datasets
Bommasani, R., & Cardie, C. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.649 · Cited by 51
“Our metrics may be repurposed to rank examples in designing curricula for curriculum learning approaches (Bengio et al., 2009).”
BackgroundKiyasseh, D., Zhu, T., & Clifton, D. A. (2021) · Nature Communications · doi:10.1038/s41467-021-24483-0 · Cited by 86
Active Clustering Ensemble With Self-Paced Learning
Zhou, P., Sun, B., Liu, X., Du, L., & Li, X. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3252586 · Cited by 58
“Since it is in line with the learning process of humans, it has been widely used in various machine learning tasks [28], [29], [30],”
BackgroundResultHypersphere-Based Remote Sensing Cross-Modal Text–Image Retrieval via Curriculum Learning
Zhang, W., Li, J., Li, S., Chen, J., Zhang, W., Gao, X., & Sun, X. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3318227 · Cited by 71
Camera-Driven Representation Learning for Unsupervised Domain Adaptive Person Re-identification
Lee, G., Lee, S., Kim, D., Shin, Y., Yoon, Y., & Ham, B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01052 · Cited by 89
“The seminal work of [2] introduces a curriculum learning strategy that trains a model using easy examples in early stages and with hard ones in later stages.”
BackgroundEliciting good teaching from humans for machine learners
Çakmak, M., & Thomaz, A. L. (2014) · Artificial Intelligence · doi:10.1016/j.artint.2014.08.005 · Cited by 69
Towards Deep Developmental Learning
Sigaud, O., & Droniou, A. (2015) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tamd.2015.2496248 · Cited by 50
Multi-view adaptive semi-supervised feature selection with the self-paced learning
Shi, C., Gu, Z., Duan, C., & Tian, Q. C. (2019) · Signal Processing · doi:10.1016/j.sigpro.2019.107332 · Cited by 54
Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation
Beyret, B., Shafti, A., & Faisal, A. A. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968488 · Cited by 81
“In this work, our main goal is to create an intelligent robotic agent that can solve manipulation tasks by learning some form of curriculum [7] as well as a structure of move-”
BackgroundCircus ANYmal: A Quadruped Learning Dexterous Manipulation with Its Limbs
Shi, F., Homberger, T., Lee, J., Miki, T., Zhao, M., Farshidian, F., Okada, K., Inaba, M., & Hutter, M. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561926 · Cited by 69
“Curriculum learning [50] is an effective strategy to solve challenging tasks, in which the task difficulties gradually increase during the learning process.”
BackgroundLearning What Data to LearnPreprint
Yang, F., Tian, F., Qin, T., Bian, J., & Liu, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08635 · Cited by 72
“A remarkable example is curriculum learning (CL) (Bengio et al., 2009) showing that a data order from easy instances to hard ones, a.k.a., a curriculum , benefits learning process.”
BackgroundResultInfluentialLearning to Navigate in Cities Without a MapPreprint
Mirowski, P., Grimes, M. K., Malinowski, M., Hermann, K. M., Anderson, K., Teplyashin, D., Simonyan, K., Kavukcuoglu, K., Zisserman, A., & Hadsell, R. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00168 · Cited by 328
“Curriculum learning gradually increases the complexity of the learning task by choosing more and more difficult examples for the learning algorithm (Bengio et al., 2009; Graves et al., 2017; Zaremba & Sutskever, 2014).”
BackgroundCo-Training of Audio and Video Representations from Self-Supervised Temporal SynchronizationPreprint
Korbar, B., Tran, D., & Torresani, L. (2018) · arXiv (Cornell University) · Cited by 47
An Information-Theoretic Perspective on Intrinsic Motivation in Reinforcement Learning: A Survey
Aubret, A., Matignon, L., & Hassas, S. (2023) · Entropy · doi:10.3390/e25020327 · Cited by 52
“This links surprise to curriculum learning [187], i.”
BackgroundA collective AI via lifelong learning and sharing at the edge
Soltoggio, A., Ben-Iwhiwhu, E., Braverman, V., Eaton, E., Epstein, B., Ge, Y., Halperin, L., How, J., Itti, L., Jacobs, M. A., et al. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00800-2 · Cited by 51
“The concept that specific previous knowledge is beneficial to learning new tasks is also exploited in curriculum learning 78,79 , where the particular sequence and complexity of the tasks are designed to facilitate learning.”
Kanda, N., Fujita, Y., Horiguchi, S., Ikeshita, R., Nagamatsu, K., & Watanabe, S. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682273 · Cited by 41
“This training pipeline is inspired by the curriculum learning concept [32] in which the neural network is first trained with an easy problem and then trained with more difficult problems.”
MethodScale-Iterative Upscaling Network for Image Deblurring
Ye, M., Lyu, D., & Chen, G. (2020) · IEEE Access · doi:10.1109/access.2020.2967823 · Cited by 58
“[32] have shown the positive effects of curriculum learning for network performance in several tasks.”
BackgroundMVSNet++: Learning Depth-Based Attention Pyramid Features for Multi-View Stereo
Chen, P.-H., Yang, H.-C., Chen, K., & Chen, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3000611 · Cited by 59
“The concept of CL is described by Bengio et al. [32] as follows: start with an easy task and then gradually increase the diffi-culties of the learning contents.”
BackgroundChang, L., Shan, L., Zhang, W., & Dai, Y. (2023) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2023.102570 · Cited by 56
DeepGate2: Functionality-Aware Circuit Representation Learning
Shi, Z., Pan, H., Khan, S., Li, M., Liu, Y., Huang, J., Zhen, H., Yuan, M., Chu, Z., & Xu, Q. (2023) · doi:10.1109/iccad57390.2023.10323798 · Cited by 63
“To train the model, we employed multi-stage training strategy, similar to training a model with an easy task and then a harder task in curriculum learning [21].”
MethodRainMamba: Enhanced Locality Learning with State Space Models for Video Deraining
Wu, H., Yang, Y., Xu, H., Wang, W., Zhou, J., & Zhu, L. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680916 · Cited by 113
“Motivated by Curriculum Learning [7], we also introduce a dynamic mechanism that facilitates the optimization by adjusting the sampling distance for positive and negative samples.”
MethodTraining Medical Image Analysis Systems like Radiologists
Maicas, G., Bradley, A. P., Nascimento, J. C., Reid, I. D., & Carneiro, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_62 · Cited by 63
“[5] proposed a new form of curriculum learning [17] that selects new tasks based not on their performance but on their performance improvement.”
BackgroundBaRC: Backward Reachability Curriculum for Robotic Reinforcement Learning
Ivanovic, B., Harrison, J., Sharma, A., Chen, M., & Pavone, M. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794206 · Cited by 74
“Our approach is based on ideas from curriculum learning, which aims to improve the rate of learning by first training on easier examples, and increasing the difficulty as training progresses until the original task becomes appropriate to train on directly [20].”
MethodAn End-to-End Language-Tracking Speech Recognizer for Mixed-Language Speech
Seki, H., Watanabe, S., Hori, T., Roux, J. L., & Hershey, J. R. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462180 · Cited by 50
“First, the model trained by the data without code-switching is a good starting point for the training of the arguably more difficult data with code-switching, in the spirit of curriculum learning [17].”
MethodYousfi, Y., Butora, J., Fridrich, J., et al. (2019) · ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) · doi:10.1145/3335203.3335727 · Cited by 51
“Curriculum Learning (CL) [3] over payload and quality factor was used to speed up the training and overcome problems with convergence.”
BackgroundMethodAccelerating Reinforcement Learning for Reaching Using Continuous Curriculum Learning
Luo, S., Kasaei, H., & Schomaker, L. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207427 · Cited by 56
“This concept has resurfaced in machine learning: curriculum learning (CL) [7], where the main idea is to train an agent to perform a defined sequence of basic tasks referred to as curriculum, and to increase the performance of agents and speed up the learning process for the final task.”
BackgroundNeural Module Networks for Reasoning over TextPreprint
Gupta, N., Lin, K., Roth, D., Singh, S., & Gardner, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.04971 · Cited by 110
“We use curriculum learning (Bengio et al., 2009) where the model is trained only on heuristically-supervised non-count questions for the first 5 epochs.”
MethodCurriculum-Based Deep Reinforcement Learning for Quantum Control
Ma, H., Dong, D., Ding, S. X., & Chen, C. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3153502 · Cited by 63
Xiao, C., Lu, P., & He, Q. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3107742 · Cited by 68
“Curriculum learning [20] is a training technique that divides the training process into a sequence of subtasks with increased difficulty levels, which is known to be able to improve the convergence by letting the agent learn on a simplified problem at the beginning stage [21].”
BackgroundKnowing What to Learn: A Metric-Oriented Focal Mechanism for Image Captioning
Ji, J., Ma, Y., Sun, X., Zhou, Y., Wu, Y., & Ji, R. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3183434 · Cited by 60
“As another series of typical training strategies, curriculum learning [71]–[73] trains machine learning models in a fixed order, from the easy samples to the hard ones.”
BackgroundLCDnet: a lightweight crowd density estimation model for real-time video surveillance
Khan, M. A., Menouar, H., & Hamila, R. (2023) · Journal of Real-Time Image Processing · doi:10.1007/s11554-023-01286-8 · Cited by 60
“The idea of curriculum learning (CL) in neural network was first presented in [23].”
BackgroundLearning from Explicit and Implicit Supervision Jointly For Algebra Word Problems
Upadhyay, S., Chang, M.-W., Chang, K.-W., & Yih, W.-T. (2016) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1029 · Cited by 52
“Similar to curriculum learning (Bengio et al., 2009), it is important to control when the algorithm starts exploring the output space using weak super-vision.”
BackgroundTextKD-GAN: Text Generation Using Knowledge Distillation and Generative Adversarial Networks
Haidar, M. A., & Rezagholizadeh, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-18305-9_9 · Cited by 70
“They used a curriculum learning strategy [2] to produce sequences of gradually increasing lengths as training progresses.”
Linear versus deep learning methods for noisy speech separation for EEG-informed attention decoding
Das, N., Zegers, J., hamme, H. V., Francart, T., & Bertrand, A. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/aba6f8 · Cited by 53
“works on segments of 100 time frames, before training on the full mixture [24], [48].”
BackgroundHierarchyNet: Hierarchical CNN-Based Urban Building Classification
Taoufiq, S., Nagy, B., & Benedek, C. (2020) · Remote Sensing · doi:10.3390/rs12223794 · Cited by 55
“formally defined curriculum based training [25].”
Making Efficient Use of Demonstrations to Solve Hard Exploration ProblemsPreprint
Paine, T. L., Gülçehre, Ç., Shahriari, B., Denil, M., Hoffman, M., Soyer, H., Tanburn, R., Kapturowski, S., Rabinowitz, N. C., Williams, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01387 · Cited by 94
“Hints can be provided to the agent which bias it towards promising regions of the state space either via reward-shaping (Ng et al., 1999) or by introducing a sequence of curriculum tasks (Bengio et al., 2009; Graves et al., 2017).”
BackgroundLearning with AMIGo: Adversarially Motivated Intrinsic Goals
Campero, A., Răileanu, R., Küttler, H., Tenenbaum, J. B., Rocktäschel, T., & Grefenstette, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12122 · Cited by 48
“Curriculum learning [3] is widely used in machine learning but the curricula are typically handcrafted which can be time consuming.”
BackgroundAccelerating Training of Transformer-Based Language Models with Progressive Layer Dropping
Zhang, M., & He, Y. (2020) · arXiv · doi:10.48550/arxiv.2010.13369 · Cited by 50
“Based on the insights from Section 3.2, and inspired by prior work on curriculum learning [41, 42] we propose a progressive schedule θ ( t ) – a temporal schedule for the expected number of ST blocks that are retained.”
MethodInfluentialDeep Pyramidal Pooling With Attention for Person Re-Identification
Martinel, N., Foresti, G. L., & Micheloni, C. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3000904 · Cited by 50
“Thus, we adopt a curriculum learning strategy [65].”
MethodTRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild
Adeli, V., Ehsanpour, M., Reid, I. D., Niebles, J. C., Savarese, S., Adeli, E., & Rezatofighi, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01314 · Cited by 88
“Second , we employ the concept of curriculum learning [9] and adopted it to train our model, which is starting with easier sub-tasks and gradually increasing the difficulty level of the tasks.”
Radhakrishnan, K., Scott, K. A., & Clausi, D. A. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3076109 · Cited by 60
“Curriculum learning is most commonly used for classification tasks where there is a disparity in the difficulty of the classes or in the difficulty within classes [16], [18], [19].”
BackgroundLearning Modular Robot Control Policies
Whitman, J. S., Travers, M. J., & Choset, H. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3284362 · Cited by 82
“To reduce training time, we used a form of curriculum learning [63], in which the multistep sequence length started is incrementally raised over the course of training.”
MethodAir Learning: a deep reinforcement learning gym for autonomous aerial robot visual navigation
Krishnan, S., Boroujerdian, B., Fu, W., Faust, A., & Reddi, V. J. (2021) · Machine Learning · doi:10.1007/s10994-021-06006-6 · Cited by 83
“an option to train the agent using curriculum learning [15].”
BackgroundOut-of-Domain Generalization From a Single Source: An Uncertainty Quantification Approach
Peng, X., Qiao, F., & Zhao, L. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3184598 · Cited by 65
“Second, we can gradually enlarge the domain transportation by increasing the uncertainty of S+ in a curriculum learning scheme [3].”
BackgroundUniversal Deep Network for Steganalysis of Color Image Based on Channel Representation
Wei, K., Luo, W., Tan, S., & Huang, J. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3196265 · Cited by 56
“2 bpnzac) and a QF of 75, we adopted the curriculum training strategy [47].”
MethodWeng, Y., Pajarinen, J., Akrour, R., Matsuda, T., Peters, J., & Maki, T. (2022) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2022.3165805 · Cited by 56
“Curriculum learning [47] is a continuous method that gradually changes the training rules from simple to complex during the agent learning process.”
BackgroundVideo description: A comprehensive survey of deep learning approaches
Rafiq, G., Rafiq, M. T., & Choi, G. S. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10414-6 · Cited by 67
Dynamic feature scaling for online learning of binary classifiers
Bollegala, D. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.05.010 · Cited by 48
“One promising criterion for selecting examples for training is curriculum learning [1].”
BackgroundCurriculum Learning Based Approaches for Noise Robust Speaker Recognition
Ranjan, S., & Hansen, J. H. L. (2017) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2017.2765832 · Cited by 61
“This learning mechanism of humans that builds upon already acquired knowledge to facilitate better assimilation of novel information has inspired a distinct category of Curriculum Learning (CL) based algorithms in Machine Learning [1].”
BackgroundMethodResultInfluentialNuclei Segmentation in Histopathological Images Using Two-Stage Learning
Kang, Q., Lao, Q., & Fevens, T. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32239-7_78 · Cited by 59
PADS: Policy-Adapted Sampling for Visual Similarity Learning
Roth, K., Milbich, T., & Ommer, B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00660 · Cited by 57
“Adaptive Learning: Curriculum Learning[4] gradually increases the difficulty of the the samples presented to the model.”
BackgroundAspect-level sentiment analysis based on gradual machine learning
Wang, Y., Chen, Q., Shen, J., Hou, B., Ahmed, M., & Li, Z. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106509 · Cited by 58
Learning Scheduling Algorithms for Data Processing ClustersPreprint
Mao, H., Schwarzkopf, M., Venkatakrishnan, S. B., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.01963 · Cited by 49
“The concept of gradually increasing job sequence length—and therefore, problem complexity—during training is a realization of curriculum learning [8] for cluster scheduling.”
BackgroundDigital Twin-based Anomaly Detection with Curriculum Learning in Cyber-physical Systems
Xu, Q., Ali, S., & Yue, T. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3582571 · Cited by 49
“This work later inspires many researchers to explore CL for various tasks such as grammar learning [30, 30, 45] and language learning [6].”
BackgroundInfluentialDHER: Hindsight Experience Replay for Dynamic Goals
Fang, M., Zhou, C., Shi, B., Gong, B., Xu, J., & Zhang, T. (2018) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 65
Kong, Y., Liu, L., Jun, W., & Tao, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00502 · Cited by 80
“The work of [2] has sparked considerable interest in applying curriculum learning to different research fields including computer vision [1, 9, 23, 30, 33, 36].”
BackgroundMethodInfluentialMachine learning-based demand forecasting in cancer palliative care home hospitalization
Soltani, M., Farahmand, M., & Pourghaderi, A. R. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104075 · Cited by 48
Zhao, F., Li, C., Wu, Z. Z., Ouyang, Y., Zhang, J., & Dai, X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.561 · Cited by 72
HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks
Ercan, B., Eker, O., Saglam, C., Erdem, A., & Erdem, E. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3372460 · Cited by 76
“To resolve this issue, we employ a curriculum learning [63] strategy during the training.”
MethodAbstraction and Generalization in Reinforcement Learning: A Summary and Framework
Ponsen, M., Taylor, M. E., & Tuyls, K. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-11814-2_1 · Cited by 61
Self-Paced Learning: An Implicit Regularization Perspective
Fan, Y., He, R., Liang, J., & Hu, B.-G. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10809 · Cited by 57
“Inspired by the learning process and cognitive mechanism of humans and animals, Bengio et al . proposed a new learning strategy called curriculum learning (CL) in [3], which gradually increases more complex samples into the training process.”
BackgroundMethodSlideRunner: A Tool for Massive Cell Annotations in Whole Slide Images
Aubreville, M., Bertram, C., Klopfleisch, R., & Maier, A. (2018) · Informatik aktuell · doi:10.1007/978-3-662-56537-7_81 · Cited by 56
Shen, Y., & Gao, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00919-9_45 · Cited by 60
“Curriculum learning is used in our training to stabilize our training process [2].”
MethodDomain Adaption in One-Shot Learning
Dong, N., & Xing, E. P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-10925-7_35 · Cited by 42
“We also discuss the how the the proposed sampling strategy is linked to distance metric learning (DML) [30] and curriculum learning [2].”
BackgroundDeep-learning continuous gravitational waves: Multiple detectors and realistic noise
Dreißigacker, C., & Prix, R. (2020) · Physical review. D/Physical review. D. · doi:10.1103/physrevd.102.022005 · Cited by 55
“This type of curriculum learning [45] is necessary to teach the network to find the weak signals at the final depth.”
BackgroundModeling and Monitoring Erosion of the Leading Edge of Wind Turbine Blades
Duthé, G., Abdallah, I., Barber, S., & Chatzi, E. (2021) · Energies · doi:10.3390/en14217262 · Cited by 49
“Here, we evaluated the transferability of the different experiments and assessed the use of curriculum learning [73] as a method to improve accuracy on difficult results.”
Entity-Relation Extraction as Multi-Turn Question AnsweringPreprint
Li, X., Yin, F., Sun, Z., Li, X., Yuan, A., Wang, G., Chai, D., Zhou, M., & Li, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05529 · Cited by 43
“For the RESUME dataset, we use the strategy of curriculum learning (Bengio et al., 2009), i.e., we gradually increase the number of turns from 2 to 4 at training.”
MethodReinforcement learning for optimization of variational quantum circuit architecturesPreprint
Ostaszewski, M., Trenkwalder, L. M., Masarczyk, W., Scerri, E., & Dunjko, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.16089 · Cited by 215
“One of the well-studied solutions to that problem, known as curriculum learning [23, 24], requires the specification of a sequence of tasks with ever-increasing level of difficulty.”
BackgroundMethodLearning Transferable Cooperative Behavior in Multi-Agent Teams
Agarwal, A., Kumar, S., Sycara, K. P., & Lewis, M. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/edcb3795 · Cited by 167
“This facilitates in establishing a curriculum (Bengio et al., 2009) of tasks with increasing difficulty.”
BackgroundLOW: Training deep neural networks by learning optimal sample weights
Santiago, C., Barata, C., Sasdelli, M., Carneiro, G., & Nascimento, J. C. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107585 · Cited by 50
“Curriculum learning was one of the first sample weighting strategies [3].”
BackgroundGREnet: Gradually REcurrent Network With Curriculum Learning for 2-D Medical Image Segmentation
Wang, J., Tang, Y., Xiao, Y., Zhou, J. T., Fang, Z., & Yang, F. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3238381 · Cited by 58
“Curriculum learning is proposed by Bengio [23], which is inspired by human learning patterns.”
BackgroundMethodInfluentialIndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality
Tang, B., Lin, M. A., Akinola, I., Handa, A., Sukhatme, G. S., Ramos, F. T., Fox, D., & Narang, Y. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.039 · Cited by 137
Chung, J., Fayyad, J., Younes, Y. A., & Najjaran, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10670-6 · Cited by 79
“To enhance the learning process, they incorporated the curriculum learning paradigm, inspired by Bengio et al. (2009), which accelerated the convergence to satisfactory solutions in a more efficient manner.”
MethodBrain-Inspired Learning, Perception, and Cognition: A Comprehensive Review
Jiao, L., Ma, M., He, P., Geng, X., Liu, X., Liu, F., Ma, W., Yang, S., Hou, B., & Tang, X. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3401711 · Cited by 62
Safety-Aware Human-in-the-Loop Reinforcement Learning With Shared Control for Autonomous Driving
Huang, W., Liu, H., Huang, Z., & Lv, C. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3420959 · Cited by 72
Generative artificial intelligence based models optimization towards molecule design enhancement
Khater, T., Alkhatib, S., Shehhi, A. A., Pitsalidis, C., Pappa, A., Ngô, S. T., Chan, V., & Truong, V. K. (2025) · Journal of Cheminformatics · doi:10.1186/s13321-025-01059-4 · Cited by 63
“…learning A training strategy where the model is initially presented with simpler and progressively more complex tasks, improving learning stability [21] Diffusion model It is a generative model that learns to gradually denoise a signal, reversing a forward process that progressively adds noise to…”
The A2iA Arabic Handwritten Text Recognition System at the Open HaRT2013 Evaluation
Bluche, T., Louradour, J., Knibbe, M., Moysset, B., BenZeghiba, M. F., & Kermorvant, C. (2014) · IAPR International Workshop on Document Analysis Systems (DAS) · doi:10.1109/das.2014.40 · Cited by 51
“Then all these RNNs were trained by running SGD on the NLL of target sequences with a constant learning rate of 0.001.”
BackgroundInfluentialSelf-citationTask-Aware Attention Model for Clothing Attribute Prediction
Zhang, S., Song, Z., Cao, X., Zhang, H., & Zhou, J. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2902268 · Cited by 50
“We employ the curriculum learning [26] strategy and a triplet framework consisting of two shop domain models and one street domain model for the street domain attribute recognition.”
MethodDeep Cross-Media Knowledge Transfer
Huang, X., & Peng, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00921 · Cited by 57
“The motivation of CL is simple: to first learn from easy samples, and gradually learn from harder samples [2], which aims to reduce the negative e ff ects brought by noisy data in early period of training.”
BackgroundPrecise proximal femur fracture classification for interactive training and surgical planning
Jiménez-Sánchez, A., Kazi, A., Albarqouni, S., Kirchhoff, C., Biberthaler, P., Navab, N., Kirchhoff, S., & Mateus, D. (2020) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-020-02150-x · Cited by 55
“Such scenarios require more complex (and thus less interpretable models) [24] or different optimization strategies [33,34].”
BackgroundThe Early Phase of Neural Network TrainingPreprint
Frankle, J., Schwab, D. J., & Morcos, A. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10365 · Cited by 164
“, 2019; Frankle & Carbin, 2019), does pre-training make it easier for sparse neural networks to learn as a rough form of curriculum learning (Bengio et al., 2009)? We experimented with training sparse sub-networks of ResNet-20 (IMP sub-networks, randomly reinitialized sub-networks, and randomly pruned sub-networks) fi…”
BackgroundAbstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution
Zhang, C., Jia, B., Zhu, S. C., & Zhu, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00961 · Cited by 79
“Based on this observation, we train our PrAE learner in a 3-stage curriculum [2].”
MethodSPFTN: A Self-Paced Fine-Tuning Network for Segmenting Objects in Weakly Labelled Videos
Zhang, D., Yang, L., Meng, D., Xu, D., & Han, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.567 · Cited by 42
“Specifically, inspired by the learning process of humans/animals, the theory of self-paced (or curriculum) learning [1, 15] is proposed in recent years.”
BackgroundA Dual-Network Progressive Approach to Weakly Supervised Object Detection
Dong, X., Meng, D., Ma, F., & Yang, Y. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123455 · Cited by 44
“Bengio et al. [3] first proposed a general learning strategy: curriculum learning (CL).”
BackgroundLearning Curriculum Policies for Reinforcement Learning
Narvekar, S., & Stone, P. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/uoaz3967 · Cited by 103
“This insight is the basis for curriculum learning [4, 15].”
BackgroundA Survey on Deep Reinforcement Learning for Data Processing and Analytics
Cai, Q., Cui, C., Xiong, Y., Wang, W., Xie, Z., & Zhang, M. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3155196 · Cited by 82
“training jobs to conduct curriculum learning [65].”
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.