Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 501–600 of 5,240
Simple to complex cross-modal learning to rank
Luo, M., Chang, X., Li, Z., Nie, L., Hauptmann, A. G., & Zheng, Q. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.07.001 · Cited by 79
“However, since this method verifies the pairwise ranking criterion with mapping from image query space to the document space, its performance may be deteriorated by the skewed multi-modal data.”
BackgroundMethodInfluentialDeep self-paced learning for person re-identification
Zhou, S., Wang, J., Meng, D., Xin, X., Li, Y., Gong, Y., & Zheng, N. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.005 · Cited by 78
“The SPL theory is inspired by the cognitive process of human beings, where samples are involved into the training process from easy to hard ones [49].”
BackgroundCurriculum-guided Hindsight Experience Replay
Fang, M., Zhou, T., Du, Y., Han, L., & Zhang, Z. (2019) · neural information processing systems · Cited by 194
Deep Learning-Based Automated Lip-Reading: A Survey
Fenghour, S., Chen, D., Guo, K., Li, B., & Xiao, P. (2021) · IEEE Access · doi:10.1109/access.2021.3107946 · Cited by 104
“This is in addition to the requirement of curriculum learning-based strategies [131], [132] which involve further pre-processing, such as the clipping of training videos with individuals speaking so that the models can be trained on single word examples to begin with, before gradually incrementing…”
BackgroundMozaffari, S., Arnold, E. H., Dianati, M., & Fallah, S. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3161785 · Cited by 97
“depending on the design of the curriculum and the datasets used [32], [33].”
Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformer
Zhang, W., Yang, L. Q., Geng, S., & Hong, S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3292066 · Cited by 152
“As a result, we introduce CL [46], which is a strategy that trains the model from “easy” to “hard.””
MethodTexture-Guided Saliency Distilling for Unsupervised Salient Object Detection
Zhou, H., Qiao, B., Yang, L.-X., Lai, J.-H., & Xie, X.-H. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52729.2023.00701 · Cited by 67
CLIP-VG: Self-Paced Curriculum Adapting of CLIP for Visual Grounding
Xiao, L., Yang, X., Peng, F., Yan, M., Wang, Y., & Xu, C. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3321501 · Cited by 114
“Our approach builds upon the general curriculum learning paradigm [47], where a model goes through multiple rounds of easy-to-hard training by leveraging its own past predictions.”
BackgroundAlgorithmic and Human Teaching of Sequential Decision Tasks
Cakmak, M., & Lopes, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8333 · Cited by 77
“For instance, curriculum design suggests that the teacher should select examples by starting with easy examples and gradually increasing the difficulty (Elman 1993; Bengio et al. 2009; Zang et al. 2010; Taylor 2009).”
BackgroundLearning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-NetworksPreprint
Foerster, J., Assael, Y., de Freitas, N., & Whiteson, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02672 · Cited by 177
“imate the Q-Values for each action Qm = MLP[128 64;64 64;64 1](ym 1 a jjyn s ). The network is trained with minibatches of 20 episodes. Furthermore, we use an adaptive variant of curriculum learning (Bengio et al., 2009) to pave the way for scalable strategies and better training performance. We sample examples from a…”
MethodDAO to HANOI via DeSci: AI Paradigm Shifts from AlphaGo to ChatGPT
Miao, Q., Zheng, W., Lv, Y., Huang, M., Ding, W., & Wang, F.-Y. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123561 · Cited by 78
“3) Curriculum Learning: The concept of Curriculum Learning [13] was first proposed by Bengio in 2009.”
BackgroundIncremental on-line adaptation of POMDP-based dialogue managers to extended domains
Gašić, M., Kim, D., Tsiakoulis, P., Breslin, C. J., Henderson, M., Szummer, M., Thomson, B., & Young, S. J. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-40 · Cited by 76
“Rather than trying to expand the full domain immediately, we gradually make the task more difficult, adopting the ideas of curriculum learning [19].”
BackgroundRobot grasping in clutter: Using a hierarchy of supervisors for learning from demonstrations
Laskey, M., Lee, J., Chuck, C., Gealy, D. V., Hsieh, W. Y.-S., Pokorny, F. T., Dragan, A. D., & Goldberg, K. Y. (2016) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/coase.2016.7743488 · Cited by 87
“Curriculum Learning Our approach is closely related to ideas from curriculum learning, where a neural network is trained incrementally: first on easier examples and then gradually on data of increasing difficulty [5].”
BackgroundLabel-Efficient Breast Cancer Histopathological Image Classification
Qi, Q., Li, Y., Wang, J., Zheng, H., Huang, Y., Ding, X., & Rohde, G. K. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2885134 · Cited by 82
“2) Confidence-Boosting Query Strategy: Both humans and animals learn much better when examples are organized in a meaningful order that gradually introduces more concepts, which progressively increase in complexity [25].”
BackgroundCross-resolution learning for Face Recognition
Massoli, F. V., Amato, G. M., & Falchi, F. (2020) · Image and Vision Computing · doi:10.1016/j.imavis.2020.103927 · Cited by 87
“In order to accomplish the task, we borrowed from the curriculum learning paradigm [29] the concept that a model can benefit from a training strategy in which more complex data are given as inputs as the learning process proceeds.”
MethodLearning in Infancy Is Active, Endogenously Motivated, and Depends on the Prefrontal Cortices
Raz, G., & Saxe, R. (2020) · Annual Review of Developmental Psychology · doi:10.1146/annurev-devpsych-121318-084841 · Cited by 73
Where to go Next: Learning a Subgoal Recommendation Policy for Navigation in Dynamic Environments
Brito, B., Everett, M., How, J. P., & Alonso–Mora, J. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3068662 · Cited by 81
“Moreover, to increase the learning speed during training, we gradually increase the number of agents in the training environments (curriculum learning [40]).”
BackgroundS2Match: Self-paced sampling for data-limited semi-supervised learning
Guan, D., Xing, Y., Huang, J., Xiao, A., Saddik, A. E., & Lu, S. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111121 · Cited by 92
Aligned Cross Entropy for Non-Autoregressive Machine Translation
Ghazvininejad, M., Karpukhin, V., Zettlemoyer, L., & Levy, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.01655 · Cited by 74
“We speculate that partially-observed inputs add easier examples to the training set, allowing for better optimization as in curriculum learning (Bengio et al., 2009).”
BackgroundA review of modularization techniques in artificial neural networks
Amer, M., & Maul, T. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-019-09706-7 · Cited by 117
“Curriculum learning (Bengio et al., 2009, 2015) is a particular form of this partitioning, where the neural network is successively trained on a sequence of subspaces with increasing complexity.”
BackgroundPedestrian intention prediction: A convolutional bottom-up multi-task approach
Razali, H., Mordan, T., & Alahi, A. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103259 · Cited by 96
Perpetual Humanoid Control for Real-time Simulated Avatars
Luo, Z., Cao, J., Winkler, A., Kitani, K., & Xu, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01000 · Cited by 381
“A paradigm has been studied in transfer learning and domain adaption as progressive learning [7, 5] or curriculum learning [1].”
BackgroundOn Efficient Training of Large-Scale Deep Learning Models
Shen, L., Sun, Y., Yu, Z., Ding, L., Tian, X., & Tao, D. (2024) · ACM Computing Surveys · doi:10.1145/3700439 · Cited by 199
“Curriculum learning [6, 44] is based on the intuition of starting with learning from simple tasks and gradually increasing the complexity of the task with some general features of data samples.”
A Survey of Autonomous Driving from a Deep Learning Perspective
Zhao, J., Wu, Y., Deng, R., Xu, S., Gao, J., & Burke, A. F. (2025) · ACM Computing Surveys · doi:10.1145/3729420 · Cited by 150
Easy Questions First? A Case Study on Curriculum Learning for Question Answering
Sachan, M., & Xing, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1043 · Cited by 70
“Curriculum learning (CL) (Bengio et al., 2009) and self-paced learning (SPL) (Kumar et al., 2010) have been recently introduced in machine learning literature.”
BackgroundLane Detection and Classification Using Cascaded CNNs
Pizzati, F., Allodi, M., Barrera, A., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-45096-0_12 · Cited by 76
“For this reason, the curriculum learning strategy [21] has been used to achieve convergence.”
MethodLip Reading Sentences Using Deep Learning With Only Visual Cues
Fenghour, S., Chen, D., Guo, K., & Xiao, P. (2020) · IEEE Access · doi:10.1109/access.2020.3040906 · Cited by 85
“The perplexity score is calculated by taking the exponentiation of the cross-entropy loss when the GPT is evaluated on a sentence and like in [27], a beam width of 50 has been used.”
BackgroundMethodCurriculum Graph Co-Teaching for Multi-Target Domain Adaptation
Roy, S., Krivosheev, E., Zhong, Z., Sebe, N., & Ricci, E. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00531 · Cited by 103
“Similarly, our CGCT is inspired by the EHTS strategy except we progressively recruit the pseudo-labeled targets [1] from the robust GCN classification head to better train the MLP classifier, which in turn regularizes the GCN head (see Sec.”
A curriculum learning method for improved noise robustness in automatic speech recognition
Braun, S., Neil, D., & Liu, S. (2017) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2017.8081267 · Cited by 81
“The noise corrupted test sets were created by mixing the clean samples with either babble noise from the NOISEX database[23] or pink noise generated by the Audacity [24] software, with each noise type added at 15 different SNR values from 50 dB to -20 dB in 5 dB steps.”
MethodAccelerating Minibatch Stochastic Gradient Descent Using Typicality Sampling
Peng, X., Li, L., & Wang, F. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2957003 · Cited by 81
Machine Learning: A Review of Learning TypesPreprint
Sah, S. (2020) · Preprints.org · doi:10.20944/preprints202007.0230.v1 · Cited by 128
“Further reading (Bengio et al., 2009).”
Interruption-Aware Cooperative Perception for V2X Communication-Aided Autonomous Driving
Ren, S., Lei, Z., Wang, Z., Dianati, M., Wang, Y., Chen, S., & Zhang, W. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3371974 · Cited by 75
“To address this challenge, we employ the concept of curriculum learning [57] wherein the learning process initially commences with easy examples and progressively escalates the task difficulty.”
MethodIn-Silico Molecular Binding Prediction for Human Drug Targets Using Deep Neural Multi-Task Learning
Lee, K., & Kim, D. (2019) · Genes · doi:10.3390/genes10110906 · Cited by 74
Curriculum Meta-Learning for Next POI Recommendation
Chen, Y., Wang, X. Y., Fan, M., Huang, J., Yang, S., & Zhu, W. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467132 · Cited by 107
PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning
Bao, S., He, H., Wang, F., Wu, H., Wang, H., Wu, W., Guo, Z., Liu, Z., & Xu, X. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.222 · Cited by 94
“In this work, we try to scale up PLATO to PLATO-2 and introduce an effective training schema via curriculum learning (Bengio et al., 2009).”
MethodVirtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction
Yin, W., Liu, Y., & Shen, C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3097396 · Cited by 105
“Several works have demonstrated that curriculum learning [43], [44], [45] can boost the performance of deep learning methods.”
BackgroundData-Centric Green Artificial Intelligence: A Survey
Salehi, S., & Schmeink, A. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3315272 · Cited by 106
“CL is a data-centric and environmentally friendly approach, as it enhances the performance of learning algorithms by improving the presentation of data without modifying the model, resulting in better solutions, improved generalization, and faster convergence [106], [107], [108], [109], [110].”
BackgroundMachine Teaching: A New Paradigm for Building Machine Learning Systems
Simard, P. Y., Amershi, S., Chickering, D. M., Pelton, A. E., Ghorashi, S., Meek, C., Ramos, G., Suh, J., Verwey, J., Wang, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.06742 · Cited by 74
“Curriculum learning (Bengio et al., 2009), for instance, could be seen as belonging squarely in the intersection because it involves both a learning algorithm and teacher behavior.”
BackgroundAgentic Large Language Models, a Survey
Plaat, A., Van Duijn, M., Van Stein, N., Preuss, M., Van der Putten, P., & Batenburg, K. J. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.18675 · Cited by 75
Progressively Trained Convolutional Neural Networks for Deformable Image Registration
Eppenhof, K. A. J., Lafarge, M. W., Veta, M., & Pluim, J. P. W. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2953788 · Cited by 82
“moving on to more difficult ones [22], [23].”
BackgroundPore, A., Li, Z., Dall’Alba, D., Hernansanz, A., Momi, E. D., Menciassi, A., Casals, A., Dankelman, J., Fiorini, P., & Poorten, E. V. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3269384 · Cited by 93
“Recently,curriculumlearninghasbeenproposed tolearninincreasinglycomplexenvironments[186].”
Deep Learning for Image Super-resolution: A SurveyPreprint
Wang, Z., Chen, J., & Hoi, S. C. H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06068 · Cited by 74
“Curriculum learning [148] refers to starting from an easier task and gradually increasing the difficulty.”
BackgroundActive visual information gathering for vision-language navigation
Wang, H., Wang, W., Shu, T., Liang, W., & Shen, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58542-6_19 · Cited by 74
“To avoid this, we adopt curriculum learning[4] to train our agent with incrementally improved exploration length.”
MethodCurriculum self-paced learning for cross-domain object detection
Soviany, P., Ionescu, R. T., Rota, P., & Sebe, N. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103166 · Cited by 78
“The general principle of the easy-to-hard training strategies known as curriculum learning [1], stems from the fact that human beings learn better when they receive easy examples first, with gradually more complex concepts being introduced later.”
BackgroundMethodInfluentialSelf-Paced Nonnegative Matrix Factorization for Hyperspectral Unmixing
Peng, J., Zhou, Y., Sun, W., Du, Q., & Xia, L. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.2996688 · Cited by 77
“SPL considers the differences in training samples by adaptively selecting samples from easy to complex into training using a self-paced regularizer [21], [22].”
MethodDual Graph enhanced Embedding Neural Network for CTR Prediction
Guo, W., Su, R., Tan, R., Guo, H., Zhang, Y., Liu, Z., Tang, R., & He, X. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467384 · Cited by 93
“To solve these issues, we design an organized learning mechanism by taking inspiration from the curriculum learning, which introduces different concepts at different time and then uses the previous learned concepts to promote the learning of new concepts [1].”
Contrastive Neural Architecture Search with Neural Architecture Comparators
Chen, Y., Guo, Y. F., Chen, Q., Li, M., Zeng, W., Wang, Y., & Tan, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00938 · Cited by 112
“Moreover, we develop a data exploration method that trains NAC in a semi-supervised way to reduce the requirement of the training data.”
MethodA Theoretical Analysis of Optimization by Gaussian Continuation
Mobahi, H., & Fisher, J. S. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9356 · Cited by 79
“…instance learning (Gehler and Chapelle 2007; Kim and Torre 2010), semi-supervised structured output (Dhillon et al. 2012), language modeling (Bengio et al. 2009), robot navigation (Pretto, Soatto, and Menegatti 2010), shape matching (Tirthapura et al. 1998), (cid:96) 0 norm minimization…”
BackgroundSelf-Training for Jointly Learning to Ask and Answer Questions
Sachan, M., & Xing, E. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1058 · Cited by 70
“In the machine learning community, this idea was introduced with the nomenclature of curriculum learning (Bengio et al., 2009), where a curriculum is designed by ranking samples based on manually curated difficulty measures.”
BackgroundMethodInfluentialConditional Generation and Snapshot Learning in Neural Dialogue Systems
Wen, T.-H., Gašić, M., Mrkšić, N., Rojas-Barahona, L. M., Su, P.-H., Ultes, S., Vandyke, D., & Young, S. J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1233 · Cited by 94
“Conceptu-ally, snapshot learning is related to curriculum learning (Bengio et al., 2009).”
BackgroundNavigation in Unknown Dynamic Environments Based on Deep Reinforcement Learning
Zeng, J., Ju, R., Qin, L., Hu, Y., Yin, Q., & Hu, C. (2019) · Sensors · doi:10.3390/s19183837 · Cited by 75
“Curriculum learning [36] is a method that gradually increases the complexity of training samples to accelerate the training process.”
MethodXiang, X., & Foo, S. Y. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030029 · Cited by 87
“In the survey [59], they mentioned the following important techniques that evaluated in ViZDoom: Wu and Tian [68] proposed a framework combining A3C with curriculum learning [69]; Parisotto and Salakhutdinov [70] developed Neural Map, a memory system with an adaptable write operator, using a spatially structured 2D me…”
MethodHaptic perception using optoelectronic robotic flesh for embodied artificially intelligent agents
Barreiros, J. A. L., Xu, A., Pugach, S., Iyengar, N., Troxell, G. W., Cornwell, A., Hong, S., Selman, B., & Shepherd, R. F. (2022) · Science Robotics · doi:10.1126/scirobotics.abi6745 · Cited by 80
“For example, curriculum learning ( 54 ), where basic skills are learned first and others are built upon them, could reduce the size of the training dataset required (e.g., learning the spatial component of temperature could be simplified by first learning to estimate touch location).”
BackgroundComputational Foundations of Natural Intelligence
van Gerven, M. A. J. (2017) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2017.00112 · Cited by 92
“This process is known as curriculum learning (Bengio et al., 2009) and is analogous to how a child learns by decomposing problems into simpler subproblems (Turing, 1950).”
BackgroundGeneralizing Question Answering System with Pre-trained Language Model Fine-tuning
Su, D., Xu, Y., Winata, G. I., Xu, P., Kim, H., Liu, Z., & Fung, P. (2019) · Proceedings of the 2nd Workshop on Machine Reading for Question Answering · doi:10.18653/v1/d19-5827 · Cited by 69
“The second method is similar to curriculum learning (Bengio et al., 2009), but because of the sparse relation among the datasets, it’s not practical to evaluate the difficulty and the degree of learning.”
MethodRecurrent neural network language model adaptation with curriculum learning
Shi, Y., Larson, M., & Jonker, C. M. (2014) · Computer Speech & Language · doi:10.1016/j.csl.2014.11.004 · Cited by 70
“Specifically, Bengio et al. (2009) shows that final ability improves if the learner is first presented ith simple input and then moved to more complex input.”
BackgroundInfluentialMa, F., Meng, D., Xie, Q., Li, Z., & Dong, X. (2017) · UTS ePRESS (University of Technology Sydney) · Cited by 110
Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection
Wang, W., Watanabe, T., Hughes, M., Nakagawa, T., & Chelba, C. (2018) · Proceedings of the Third Conference on Machine Translation: Research Papers · doi:10.18653/v1/w18-6314 · Cited by 67
“es not seem to improve. Previous work on domain data selection has shown that the order in which data are scheduled matters a lot for NMT training, a research that is relevant to curriculum learning (Bengio et al., 2009) in machine learning literature. (van der Wees et al., 2017) show the effectiveness of a nice “grad…”
BackgroundInfluentialCyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation
Tian, Y., Hu, D., & Xu, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00277 · Cited by 108
“To this end, we learn our CCoL model with three steps in a curriculum learning [4] manner.”
MethodUncertainty-guided Model Generalization to Unseen Domains
Qiao, F., & Peng, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00672 · Cited by 96
“Second, we can gradually enlarge the domain transportation by increasing the uncertainty of S in a curriculum learning scheme [2].”
BackgroundPaheding, S., Saleem, A. I., Siddiqui, M. F. H., Rawashdeh, N. A., Essa, A. E., & Reyes-Angulo, A. A. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10165-7 · Cited by 105
“We also suggest exploring other types of machine learning techniques like curriculum learning [331], where the model is trained on easier tasks or samples first, gradually increasing the complexity to improve learning efficiency and performance.”
Deep belief networks and cortical algorithms: A comparative study for supervised classification
Rizk, Y., Hajj, N., Mitri, N., & Awad, M. (2018) · Applied Computing and Informatics · doi:10.1016/j.aci.2018.01.004 · Cited by 80
Training highly multiclass classifiers
Gupta, M. R., Bengio, S., & Weston, J. (2014) · Journal of Machine Learning Research · doi:10.5555/2627435.2638582 · Cited by 101
“a form of curriculum learning, that is, of learning to distinguish easy classes before focusing on learning to distinguish hard classes (Bengio et al., 2009).”
BackgroundSelf-citationBuilding medical image classifiers with very limited data using segmentation networks
Wong, K. C. L., Syeda-Mahmood, T. F., & Moradi, M. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.07.010 · Cited by 73
“Although fundamental analysis such as that in Bengio et al. (2009) exists, it is very challenging to develop a similar analysis for our framework.”
BackgroundDuan, Y., Zhu, H., Wang, H., Yi, L., Nevatia, R., & Guibas, L. J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_4 · Cited by 111
Attention-Based Deep Learning Framework for Human Activity Recognition With User Adaptation
Buffelli, D., & Vandin, F. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3067690 · Cited by 68
“Future research directions include the optimization of the personalization process to minimize the feedback required from the user, for example by using data augmentation or curriculum training techniques [5].”
BackgroundSelf-paced and self-consistent co-training for semi-supervised image segmentation
Wang, P., Peng, J., Pedersoli, M., Zhou, Y., Zhang, C., & Desrosiers, C. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102146 · Cited by 85
“Self-paced learning: Self-paced learning (SPL) [32] extends the paradigm of curriculum learning [33], where a model is learned by adding gradually more difficult instances during training.”
BackgroundProgressive Transfer Learning for Face Anti-Spoofing
Quan, R., Wu, Y., Yu, X., & Yang, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3066912 · Cited by 78
“[19] proposed an easy-to-hard learning strategy”
Learning from the Master: Distilling Cross-modal Advanced Knowledge for Lip Reading
Ren, S., Du, Y., Lv, J., Han, G., & He, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01312 · Cited by 70
“Therefore, we incorporate a curriculum learning [4, 13] based strategy into the training stage of our approach.”
MethodProgressive Meta-Learning With Curriculum
Zhang, J., Song, J., Gao, L., Liu, Y., & Shen, H. T. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3164190 · Cited by 78
Dual-Evidential Learning for Weakly-supervised Temporal Action Localization
Chen, M., Gao, J., Yang, S., & Xu, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19772-7_12 · Cited by 108
“Inspired by Curriculum Learning [4], we propose a progressive learning method by leveraging snippet-level uncertainty to help the model learn the entire action instance progressively and comprehensively.”
MethodA Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to ImitationPreprint
Tai, L., Zhang, J., Liu, M., Boedecker, J., & Burgard, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.07139 · Cited by 143
“Finn et al. (2017b) and Yu et al. (2018) both extend the Model-Agnostic Meta-Learning (MAML) method (Finn et al., 2017a), which we will briefly review here before proceeding.”
InfluentialChallenges in End-to-End Neural Scientific Table Recognition
Deng, Y., Rosenberg, D. S., & Mann, G. (2019) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2019.00148 · Cited by 93
“Curriculum learning may help accelerate training [55].”
Soil-Adaptive Excavation Using Reinforcement Learning
Egli, P., Gaschen, D., Kerscher, S., Jud, D., & Hutter, M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3189834 · Cited by 97
“To facilitate learning, we make it at the beginning of the training easier for the agent to achieve the desired final state by introducing a curriculum ( k j ) [39], which is changed based on the RL update count ( j ).”
Reinforcement Learning based Curriculum Optimization for Neural Machine Translation
Kumar, G., Foster, G. F., Cherry, C., & Krikun, M. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1208 · Cited by 86
“The idea of a curriculum was popularized by Bengio et al. (2009), who viewed it as a way to improve convergence by presenting heuristicallyidentified easy examples first.”
BackgroundHWNet v2: an efficient word image representation for handwritten documents
Krishnan, P., & Jawahar, C. V. (2019) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-019-00336-x · Cited by 88
“To avoid such scenarios, we use the strategy from curriculum learning [8], where we start the training process from synthetic images showing easy examples (based on the number of characters) first and harder later.”
MethodHagendorff, T., Dasgupta, I., Binz, M., Chan, S. C. Y., Lampinen, A., Wang, J. X., Akata, Z., & Schulz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13988 · Cited by 68
Open-Ended Learning: A Conceptual Framework Based on Representational Redescription
Doncieux, S., Filliat, D., Díaz-Rodríguez, N., Hospedales, T. M., Duro, R. J., Coninx, A., Roijers, D. M., Girard, B., Perrin, N., & Sigaud, O. (2018) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00059 · Cited by 93
“The efficiency of a learning system is influenced by the order of the tasks it is facing (Bengio et al., 2009).”
Curriculum-Meta Learning for Order-Robust Continual Relation Extraction
Wu, T., Li, X., Li, Y.-F., Haffari, G., Qi, G., Zhu, Y., & Xu, G. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17241 · Cited by 93
“In this paper, we tackle this problem by leveraging a curriculum learning method (Bengio et al. 2009).”
MethodV2VFormer++: Multi-Modal Vehicle-to-Vehicle Cooperative Perception via Global-Local Transformer
Yin, H., Tian, D., Lin, C., Duan, X., Zhou, J., Zhao, D., & Cao, D. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3314919 · Cited by 90
“For OPV2V [19], we introduce the curriculum learning strategy [60] to imitate the human cognition mechanism: the model is trained for 35 epochs at sim mode and another 10 epochs with real setting (e.g., localization error, async overhead, etc.), optimized by Adam [61] with 0 .”
MethodUnified Multi-Modal Image Synthesis for Missing Modality Imputation
Zhang, Y., Peng, C., Wang, Q., Song, D., Li, K., & Zhou, S. K. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3424785 · Cited by 126
“However, since different missing data imputation tasks pose varying levels of difficulty, we introduce a Curriculum Learning [32] based training scheme.”
MethodCurriculum learning for ab initio deep learned refractive optics
Yang, X., Fu, Q., & Heidrich, W. (2024) · Nature Communications · doi:10.1038/s41467-024-50835-7 · Cited by 92
“In this work, we eliminate the need for good starting points and continuous manipulation in the lens design process by introducing an automatic method based on curriculum learning [30, 31, 32].”
MethodQiao, Z., Muelling, K., Dolan, J. M., Palanisamy, P., & Mudalige, P. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500603 · Cited by 72
“Instead of training an agent from scratch, here we suggest using curriculum learning (CL)[21] by first choosing a relatively easy task to train on and then gradually increasing the difficulty of the problem presented to the agent during the learning process.”
BackgroundMethodMorerio, P., Cavazza, J., Volpi, R., Vidal, R., & Murino, V. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.383 · Cited by 107
“For the sake of clarity, let us remind the concept of curriculum learning [2].”
BackgroundMethodInfluentialFan, T., Long, P., Liu, W., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.03841 · Cited by 68
“Inspired by the curriculum learning paradigm (Bengio, Louradour, Collobert, & Weston, 2009), we propose a two-stage training process, which accelerates the policy to converge to a satisfactory solution, and gets higher rewards than the policy trained from scratch with the same number of epoch (as…”
MethodEmergent behaviour and neural dynamics in artificial agents tracking odour plumes
Singh, S. H., van Breugel, F., Rao, R. P. N., & Brunton, B. W. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00599-w · Cited by 61
“More complex training curricula 81 or alternative training algorithms using evolutionary techniques 82 might be able to mitigate the notable performance variability we observed in our agents.”
BackgroundTENet: Triple Excitation Network for Video Salient Object Detection
Ren, S., Han, C., Yang, X., Han, G., & He, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58558-7_13 · Cited by 81
“This strategy shares a similar spirit to the curriculum learning framework [2], in which they argue that training with an easy task at first then continues with more difficult tasks gradually may leads to better optimization.”
MethodOn the Link between Gaussian Homotopy Continuation and Convex Envelopes
Mobahi, H., & Fisher, J. S. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-14612-6_4 · Cited by 95
“Other examples that utilize continuation for optimization are clustering [32], graph matching [31,79,42], multiple instance learning [37], language modeling [3], manifold sampling [58], and `0 norm minimization [65].”
BackgroundSource Task Creation for Curriculum Learning
Narvekar, S., Sinapov, J., Leonetti, M., et al. (2016) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.5555/2936924.2937007 · Cited by 67
Webly Supervised Learning of Convolutional NetworksPreprint
Chen, X., & Gupta, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.01554 · Cited by 67
“After seventh layer, another fully connected layer is used to predict class labels.”
MethodMisra, I., Girshick, R., Fergus, R., Hebert, M., Gupta, A., & van der Maaten, L. J. P. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00009 · Cited by 86
“Inspired by curriculum learning [5, 6, 29, 45], we use this difference to select questions on which the answering module can improve the fastest.”
MethodUnsupervised Multi-modal Style Transfer for Cardiac MR Segmentation
Chen, C., Ouyang, C., Tarroni, G., Schlemper, J., Qiu, H., Bai, W., & Rueckert, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-39074-7_22 · Cited by 84
“Inspired by curriculum learning [8] and transfer learning, we first train a segmentation network using annotated bSSFP images (source domain; easy examples) and then fine-tune it to segment LGE images (target domain; hard examples).”
Espresso: A Fast End-to-End Neural Speech Recognition Toolkit
Wang, Y., Chen, T., Xu, H., Ding, S., Lv, H., Shao, Y., Peng, N., Xie, L., Watanabe, S., & Khudanpur, S. P. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003968 · Cited by 85
“Curriculum learning [50], which is quite helpful to stabilize training with long sequences (e.”
BackgroundLi, B., Keikhosravi, A., Loeffler, A. G., & Eliceiri, K. W. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101938 · Cited by 71
Back, S., Lee, S., Shin, S., Yu, Y., Yuk, T., Jong, S., Ryu, S., & Lee, K. (2021) · IEEE Access · doi:10.1109/access.2021.3054403 · Cited by 76
“Motivated from [53], which changes the training dataset from a low-level-knowledge to a high-level knowledge dataset, we sequentially updated the teacher network used for KD from a low-performance to a high-performance network.”
MethodRefign: Align and Refine for Adaptation of Semantic Segmentation to Adverse Conditions
Brüggemann, D., Sakaridis, C., Truong, P., & Gool, L. V. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00319 · Cited by 95
“In line with the principle of curriculum learning [3], the model should thus rely more heavily on the “easier” reference images first.”
MethodCMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning
Shu, J., Yuan, X., Meng, D., & Xu, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3271451 · Cited by 93
“Establishing Meta Dataset: Motivated by curriculum learning [31], [90], [91], we select the most confident training samples as meta data.”
MethodWeakly Supervised Object Localization Using Size Estimates
Shi, M., & Ferrari, V. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_7 · Cited by 67
“A strong assumption in [22] is that the curriculum is provided by a human teacher.”
BackgroundInfluentialPseudo-Labeling Curriculum for Unsupervised Domain AdaptationPreprint
Choi, J., Jeong, M., Kim, T., & Kim, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.00262 · Cited by 112
“The curriculum learning [2] is motivated by the cognitive process of humans that gradually learn from easy to complex samples.”
Lessons from infant learning for unsupervised machine learning
Zaadnoordijk, L., Besold, T. R., & Cusack, R. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00488-2 · Cited by 65
“171 and was prominently addressed in the deep learning literature 172 .”
Background
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.