Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 901–1,000 of 5,240
Learning Representations for Facial Actions From Unlabeled Videos
Li, Y., Zeng, J., & Shan, S. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3011063 · Cited by 51
Tang, P., Yan, X., Liang, Q., & Zhang, D. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107656 · Cited by 41
UniCon: Unified Context Network for Robust Active Speaker Detection
Zhang, Y., Liang, S., Yang, S., Liu, X., Wu, Z., Shan, S., & Chen, X. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475275 · Cited by 54
“To facilitate training, we apply curriculum learning [3] by first training a single-candidate model without relational context according to Eq.”
Ben-Yosef, M., & Weinshall, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.10356 · Cited by 71
CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement LearningPreprint
Yang, J., Nakhaei, A., Isele, D., Fujimura, K., & Zha, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.05188 · Cited by 69
“To the best of our knowledge, few have investigated the benefits of curriculum learning [2] for MARL.”
BackgroundUncertainty-aware Self-training for Text Classification with Few LabelsPreprint
Mukherjee, S., & Awadallah, A. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.15315 · Cited by 61
“One of the earlier works in neural networks leveraging easiness of the samples for learning is given by curriculum learning [Bengio et al., 2009].”
BackgroundCore Challenges in Embodied Vision-Language Planning
Francis, J., Kitamura, N., Labelle, F., Lu, X., Navarro, I., & Oh, J. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13646 · Cited by 62
“Babywalk then uses curriculum-based RL to refine the policy by giving the agent increasingly longer navigation tasks (Bengio et al., 2009).”
MethodPhone-to-Audio Alignment without Text: A Semi-Supervised Approach
Zhu, J. Y., Zhang, C., & Jurgens, D. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746112 · Cited by 79
“To facilitate alignment learning, we train the model incrementally in the curriculum learning paradigm [25] on increasingly longer audio samples.”
MethodDual Contrastive Learning Network for Graph Clustering
Peng, X., Cheng, J., Tang, X., Liu, J., & Wu, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3244397 · Cited by 53
“Curriculum learning [41] assigns different weights to measure the importance of samples.”
BackgroundAnnealed gradient descent for deep learning
Pan, H., Niu, X., Li, R., Dou, Y., & Jiang, H. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.11.021 · Cited by 42
“Recently, deep learning (Bengio, 2009) has achieved huge successes in many real-world applications, such as speech recognition and computer vision.”
BackgroundClassification of PolSAR Images Using Multilayer Autoencoders and a Self-Paced Learning Approach
Chen, W., Gou, S., Wang, X., Li, X., & Jiao, L. (2018) · Remote Sensing · doi:10.3390/rs10010110 · Cited by 39
“SPL can trace back to curriculum learning (CL) [19] proposed by Bengio et al.”
BackgroundWeakly-supervised object detection via mining pseudo ground truth bounding-boxes
Zhang, Y., Bai, Y., Ding, M., Li, Y., & Ghanem, B. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.07.005 · Cited by 39
An Appraisal of Nodule Diagnosis for Lung Cancer in CT Images
Zhang, G., Yang, Z., Gong, L., Jiang, S., Wang, L., Cao, X., Wei, L., Zhang, H., & Liu, Z. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1327-0 · Cited by 42
Continual Occlusion and Optical Flow Estimation
Neoral, M., Šochman, J., & Matas, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20870-7_10 · Cited by 47
Convolutional Neural Networks With Dynamic Regularization
Wang, Y., Bian, Z.-P., Hou, J., & Chau, L. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2997044 · Cited by 51
“, the curriculum learning [14], [21] or selfpaced learning [26]).”
BackgroundChen, G., Yao, S., Ma, J., Pan, L., Chen, Y., Xu, P., Ji, J., & Chen, X. (2020) · Sensors · doi:10.3390/s20174836 · Cited by 47
“Curriculum learning [54] aims to improve learning performance by designing appropriate curriculums for progressive learners from simple to difficult.”
BackgroundEmbodied Visual Navigation With Automatic Curriculum Learning in Real Environments
Morad, S., Mecca, R., Poudel, R. P. K., Liwicki, S., & Cipolla, R. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3048662 · Cited by 62
“Curriculum for neural networks was proposed by [26], and [27] affords a thorough overview of ACL applied to DRL.”
MethodDooraki, A. R., & Lee, D. J. (2020) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2020.103671 · Cited by 43
Self-Paced Learning for Neural Machine Translation
Wan, Y., Yang, B., Wong, D. F., Zhou, Y., Chao, L. S., Zhang, H., & Chen, B. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.80 · Cited by 61
“As an alternative mitigation, curriculum learning (CL, Elman, 1993; Bengio et al., 2009) has shown its effectiveness on speeding up the convergence and stabilizing the NMT model training (Zhang et al.”
BackgroundModeling multi-species RNA modification through multi-task curriculum learning
Xiong, Y., He, X., Zhao, D. Y., Tian, T., Hong, L., Jiang, T., & Zeng, J. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkab124 · Cited by 48
“To address this problem, we trained all the tasks sequentially following the strategy of curriculum learning which is a type of learning starting from relatively easy tasks and gradually increasing task difficulty (35,36).”
MethodInfluentialContrastive Curriculum Learning for Sequential User Behavior Modeling via Data Augmentation
Bian, S., Zhao, W. X., Zhou, K., Cai, J., He, Y., Yin, C., & Wen, J.-R. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3481905 · Cited by 63
“Actually, it has been shown the schedule strategy of learning tasks in varied difficulty levels directly affect the performance of the learner [1, 27].”
BackgroundMethodInfluentialProgressive Self-Training with Discriminator for Aspect Term Extraction
Wang, Q., Wen, Z., Zhao, Q., Yang, M., & Xu, R. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.23 · Cited by 43
“Curriculum Learning Learning from easier samples first and harder samples later is a common strategy in curriculum learning (Bengio et al., 2009).”
BackgroundMethodInfluentialChallenges of Neural Machine Translation for Short Texts
Wan, Y., Yang, B., Wong, D. F., Chao, L. S., Yao, L., Zhang, H., & Chen, B. (2022) · Computational Linguistics · doi:10.1162/coli_a_00435 · Cited by 47
Pilault, J., Elhattami, A., & Pal, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.09139 · Cited by 95
“As opposed to Curriculum Learning (Bengio et al., 2009), MT-Uncertainty dynamically prioritizes the most difficult tasks.”
BackgroundChaffre, T., Moras, J., Chan-Hon-Tong, A., & Marzat, J. (2020) · Proceedings of the 17th International Conference on Informatics in Control, Automation and Robotics · doi:10.5220/0009821603140323 · Cited by 38
“We propose a new learning strategy to tackle this problem, similar to Curriculum Learning (CL) (Elman, 1993; Bengio et al., 2009) but easier to implement.”
MethodFedDSR: Daily Schedule Recommendation in a Federated Deep Reinforcement Learning Framework
Huang, W. X., Liu, J., Li, T., Huang, T., Ji, S., & Wan, J. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3130265 · Cited by 46
“Curriculum learning (CL) was proposed by Bengio et al. [43] in 2009, and the primary concept is to mimic the characteristics of human learning by studying courses from easy to dif-ficult.”
BackgroundA Survey on Classifying Big Data with Label Noise
Johnson, J. M., & Khoshgoftaar, T. M. (2022) · Journal of Data and Information Quality · doi:10.1145/3492546 · Cited by 53
“This study mitigates overfitting to corrupted labels by employing techniques from Curriculum Learning (CL) [9], where models are introduced to samples in a meaningful order that facilitates learning.”
MethodCurriculum-Based Reinforcement Learning for Distribution System Critical Load Restoration
Zhang, X., Eseye, A. T., Knueven, B., Liu, W., Reynolds, M. J., & Jones, W. (2022) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2022.3209919 · Cited by 50
“As demonstrated in [22], CL can 1) accelerate policy convergence and 2) improve the quality of the obtained local-optimum in the non-convex setting.”
BackgroundData-Driven Short-Term Daily Operational Sea Ice Regional Forecasting
Grigoryev, T., Verezemskaya, P., Krinitskiy, M., Anikin, N., Gavrikov, A., Trofimov, I., Balabin, N., Shpilman, A., Eremchenko, A., Gulev, S., et al. (2022) · Remote Sensing · doi:10.3390/rs14225837 · Cited by 44
GSC: A Graph and Spatio-Temporal Continuity Based Framework for Accident Anticipation
Wang, T., Chen, K., Chen, G., Li, B., Li, Z., Liu, Z., & Jiang, C. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3257169 · Cited by 59
“Therefore, inspired by curriculum learning [44], Kataoka et al. propose an AdaLEA loss for accident prediction, which allows a model to gradually learn earlier anticipation as training progresses.”
BackgroundMultiscale Adaptive Fusion Network for Hyperspectral Image Denoising
Pan, H., Gao, F., Dong, J., & Du, Q. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3257051 · Cited by 43
“Following the easy-to-difficult learning strategy in Curriculum learning [68], we incrementally”
Autonomous Vehicle Decision-Making and Control in Complex and Unconventional Scenarios—A Review
Sana, F., Azad, N. L., & Raahemifar, K. (2023) · Machines · doi:10.3390/machines11070676 · Cited by 55
“Curriculum learning was first introduced in [64] to speed up the learning process by first training a model with a simpler task and gradually increasing the complexity of the problem.”
BackgroundMethodHong, J. H., Kim, M., Choi, J., & Ro, Y. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01801 · Cited by 94
“We adopt curriculum learning [71].”
FDNet: Fourier transform guided dual-channel underwater image enhancement diffusion network
Zhu, Z., Li, X., Ma, Q., Zhai, J., & Hu, H. (2024) · Science China Technological Sciences · doi:10.1007/s11431-024-2824-x · Cited by 46
Curriculum Learning for Facial Expression Recognition
Gui, L., Baltrušaitis, T., & Morency, L. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.68 · Cited by 50
“Such sampling strategy of mini-batches fails to consider the complexity of image samples where noisy or difficult samples can slow down convergence or even mislead the learner rather than help it establish the right location of the decision surface [6].”
BackgroundInfluentialShen, R., Yan, K., Tian, K., Jiang, C., & Zhou, K. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.07.013 · Cited by 37
MetaMT, a Meta Learning Method Leveraging Multiple Domain Data for Low Resource Machine Translation
Li, R., Wang, X., & Yu, H. Q. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6339 · Cited by 47
“As proved in previous work (Bengio et al. 2009), adjusting the order of training data leads to improvements in both the convergence speed and performances.”
Towards RL-Based Hydraulic Excavator Automation
Egli, P., & Hutter, M. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341598 · Cited by 49
“kc,1 and kc,2 are curriculum factors which make the learning task easier at the beginning of the training to accelerate conversion [19].”
BackgroundDeep Reinforcement Learning for Navigation in AAA Video Games
Alonso, E., Peter, M., Goumard, D., & Romoff, J. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/294 · Cited by 65
“To allow the agent to have informative trajectories at all stages during training, we use a training curriculum [Bengio et al., 2009] and increase the radius of the spawning cylinder until the full map is covered.”
MethodNeural ordinary differential equation control of dynamics on graphs
Asikis, T., Böttcher, L., & Antulov-Fantulin, N. (2022) · Physical Review Research · doi:10.1103/physrevresearch.4.013221 · Cited by 52
“3) in a curriculum learning procedure [15], where NODEC is initially trained on trajectories sampled for low values of T .”
BackgroundReinforcement learning of rare diffusive dynamics
Das, A., Rose, D. C., Garrahan, J. P., & Limmer, D. T. (2021) · The Journal of Chemical Physics · doi:10.1063/5.0057323 · Cited by 58
Speech Emotion Recognition Using Sequential Capsule Networks
Wu, X., Cao, Y., Lu, H., Liu, S., Wang, D., Wu, Z., Liu, X., & Meng, H. M. L. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3120586 · Cited by 41
“Hence, we introduce a simple, yet effective curriculum learning (CL)-based training scheme to the capsule training [45].”
Direction of Arrival Estimation of Sound Sources Using Icosahedral CNNs
Díaz-Guerra, D., Miguel, A., & Beltrán, J. R. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3224282 · Cited by 38
“Similar to the curriculum learning [47] strategy employed in [27], we keep fixed the SNR of the simulations to 30 dB during the first 25 epochs and then we employed uniformly distributed random values from 5 dB to 30 dB in the following epochs.”
MethodAre all training examples equally valuablePreprint
Lapedriza, À., Pirsiavash, H., Bylinskii, Z., & Torralba, A. B. (2013) · arXiv (Cornell University) · doi:10.13016/m2jp7z-giib · Cited by 84
Deep Implicit Coordination Graphs for Multi-agent Reinforcement LearningPreprint
Li, S., Gupta, J. K., Morales, P., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11438 · Cited by 38
“[19] combined a two-stage attention network with a graph neural network for communication between the agents to achieve state-of-the-art performance on the traffic junction domain with curriculum training [2].”
BackgroundMethodGraphFormers: GNN-nested Transformers for Representation Learning on Textual GraphPreprint
Yang, J., Liu, Z., Xiao, S., Li, C., Lian, D., Agrawal, S., Singh, A., Sun, G., & Xie, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.02605 · Cited by 255
“As such, GraphFormers may end up with insufficiently trained GNNs. Inspired by recent success of curriculum learning (Bengio et al., 2009), we propose to train the model progressively : the first round of training is performed with manipulated data, where the nodes are randomly polluted; thus, it…”
BackgroundCost-Effective Object Detection: Active Sample Mining With Switchable Selection Criteria
Wang, K., Lin, L., Yan, X., Chen, Z., Zhang, D., & Zhang, L. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2852783 · Cited by 38
“Inspired by the principles of human/animal cognition, CL [15] was the first machine learning paradigm to adopt the concept of gradually adding samples to the training data in a controlled and meaningful (e.”
BackgroundMethodBell, P., & Renals, S. J. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178780 · Cited by 36
“A second, more closely-related method is curriculum learning [22]: the idea here is to first present the network with “easier” examples, illustrating simpler concepts, and gradually increasing the complexity: in other words, initially training on samples from a lower-entropy distribution and increasing entropy during…”
Zhang, C., Fan, W., Wang, B., Chen, C., & Li, H. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102345 · Cited by 49
Solving Combinatorial Optimization Problems with Deep Neural Network: A Survey
Wang, F., He, Q., & Li, S. (2024) · Tsinghua Science & Technology · doi:10.26599/tst.2023.9010076 · Cited by 39
SceneFun3D: Fine-Grained Functionality and Affordance Understanding in 3D Scenes
Delitzas, A., Takmaz, A., Tombari, F., Sumner, R. W., Pollefeys, M., & Engelmann, F. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01377 · Cited by 131
“Inspired by the concept of curriculum learning [5], we start training with coarse instance masks at first, which are easier for the network to detect.”
MethodA review of named entity recognition: from learning methods to modelling paradigms and tasks
Seow, W. L., Chaturvedi, I., Hogarth, A., Mao, R., & Cambria, E. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11321-8 · Cited by 75
I2VM: Incremental import vector machines
Roscher, R., Förstner, W., & Waske, B. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.04.004 · Cited by 40
“Though resulting in an output lying in the range [0,1], this does not imply that the transformed output is a good approximation of the posterior, i.”
BackgroundA self-paced learning algorithm for change detection in synthetic aperture radar images
Shang, R., Yuan, Y., Jiao, L., Yang, M., & E., A. M. G. (2017) · Signal Processing · doi:10.1016/j.sigpro.2017.07.023 · Cited by 37
“[45] proposed curriculum learning raising widespread concern in machine learning and computer vision fields.”
BackgroundEnd-to-end Multimodal Speech Recognition
Palaskar, S., Sanabria, R., & Metze, F. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462439 · Cited by 37
“We use curriculum learning [31] for the first epoch to speed up convergence.”
MethodObject-Oriented Curriculum Generation for Reinforcement Learning
da Silva, F. L., & Costa, A. H. R. (2018) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/insi6337 · Cited by 63
“More recently, inspired by the Curriculum Learning approach initially applied for Supervised Learning methods [2], Curricula started to be employed for RL agents.”
MethodTask Refinement Learning for Improved Accuracy and Stability of Unsupervised Domain Adaptation
Ziser, Y., & Reichart, R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1591 · Cited by 36
“Our approach is inspired by curriculum learning (CL (Elman, 1993; Bengio et al., 2009)), a learning paradigm that advocates the presentation of training examples to a learning algorithm in an organized manner, so that more complex concepts are learned after simpler ones.”
MethodInfluentialScheduled sampling for one-shot learning via matching network
Zhang, L., Liu, J., Luo, M., Chang, X., Zheng, Q., & Hauptmann, A. G. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.07.007 · Cited by 36
Audio Steganalysis with Improved Convolutional Neural Network
Lin, Y., Wang, R., Yan, D., Dong, L., & Zhang, X. (2019) · ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) · doi:10.1145/3335203.3335736 · Cited by 41
“Inspired by curriculum learning [1] and transfer learning strategy [19], we further fine-tune our network to solve above problems.”
MethodUsing deep learning to localize gravitational wave sources
Chatterjee, C., Wen, L., Vinsen, K., Kovalam, M., & Datta, A. (2019) · Physical review. D/Physical review. D. · doi:10.1103/physrevd.100.103025 · Cited by 45
“We considered three different SNR ranges in our experiments with signal plus Gaussian noise : [50-55], [20-35] and [10-110].”
BackgroundMethodSPLBoost: An Improved Robust Boosting Algorithm Based on Self-Paced Learning
Wang, K., Wang, Y., Zhao, Q., Meng, D., Liao, X., & Xu, Z. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2957101 · Cited by 44
“[37] proposed a new learning paradigm named curriculum learning which is the origin of SPL.”
BackgroundModular Robot Design Synthesis with Deep Reinforcement Learning
Whitman, J. S., Bhirangi, R., Travers, M. J., & Choset, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i06.6611 · Cited by 39
“We use curriculum learning (Bengio et al. 2009) on the obstacle grid, mass penalty, and complexity penalty.”
MethodRobot Navigation with Map-Based Deep Reinforcement Learning
Chen, G., Pan, L., Chen, Y., Xu, P., Wang, Z., Wu, P., Ji, J., & Chen, X. (2020) · IEEE International Conference on Networking, Sensing and Control (ICNSC) · doi:10.1109/icnsc48988.2020.9238090 · Cited by 52
“Curriculum learning [24] is a learning strategy in machine learning, which starts from easy instances and then gradually handles harder ones.”
BackgroundInclined Quadrotor Landing using Deep Reinforcement Learning
Kooi, J. E., & Babuška, R. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636096 · Cited by 53
“• Building upon the state-of-the-art model-free proximal policy optimization (PPO) algorithm [13], we propose a powerful curriculum learning [14] approach to facilitate convergence when using sparse rewards, without the need for applying a multi-goal setting like in hindsight experience replay [15] or iterated supervi…”
MethodDiagnosing glaucoma on imbalanced data with self-ensemble dual-curriculum learning
Zhao, R., Chen, X., Chen, Z., & Li, S. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102295 · Cited by 39
“Curriculum learning: Curriculum learning (Bengio et al., 2009) represents a learning regime inspired by the learning proceeds of humans that gradually proceeds from easy to more complex or hard to deal with the samples imbalance.”
BackgroundAutomated Curriculum Learning for Neural NetworksPreprint
Graves, A., Bellemare, M. G., Menick, J., Munos, R., & Kavukcuoglu, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.03003 · Cited by 644
“However curriculum learning has only recently become prevalent in the field (e.g., Bengio et al., 2009), due in part to the greater complexity of problems now being considered.”
BackgroundSource-Free Progressive Graph Learning for Open-Set Domain Adaptation
Luo, Y., Wang, Z., Chen, Z., Huang, Z. H., & Baktashmotlagh, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3270288 · Cited by 59
Feature Separation and Recalibration for Adversarial Robustness
Kim, W. J., Cho, Y., Jung, J., & Yoon, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00791 · Cited by 63
“Inspired by curriculum learning [5], CAT [8] and FAT [60] trained models with increasingly stronger adversarial examples to improve generalization.”
BackgroundMethodDistributed Deep Joint Source-Channel Coding over a Multiple Access Channel
Yilmaz, S. F., Karamanlı, C., & Gündüz, D. (2023) · International Conference on Communications · doi:10.1109/icc45041.2023.10279499 · Cited by 43
“CL is a progressive training paradigm, where network is first trained on easy tasks and then adapted to the main task of interest [22].”
BackgroundCalzada-García, A., Victores, J. G., Naranjo-Campos, F. J., & Balaguer, C. (2025) · Algorithms · doi:10.3390/a18010023 · Cited by 63
“In these two cases, the importance of curriculum learning [96] to obtain better results is highlighted.”
Evolving Culture Versus Local Minima
Bengio, Y. (2014) · Studies in computational intelligence · doi:10.1007/978-3-642-55337-0_3 · Cited by 43
Self-citationVariational Autoencoded Regression: High Dimensional Regression of Visual Data on Complex Manifold
Yoo, Y., Yun, S., Chang, H. J., Demiris, Y., & Choi, J. Y. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.314 · Cited by 34
“Mini-Batch Training: The work in [7] reports that the composition of the mini-batch is critical when using variants of stochastic gradient descent methods [23, 10] to train the parameters.”
MethodEvaluation Metrics for Machine Reading Comprehension: Prerequisite Skills and Readability
Sugawara, S., Kido, Y., Yokono, H., & Aizawa, A. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1075 · Cited by 51
“The datasets can then be ordered according to the grades of the metrics and applied to each step of the development, as in curriculum learning (Bengio et al., 2009) and transfer learning (Pan and Yang, 2010).”
MethodVariational Bayesian Multiple Instance Learning with Gaussian Processes
Haußmann, M., Hamprecht, F. A., & Kandemir, M. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.93 · Cited by 45
“Following the observation that weakly supervised localization algorithms often have more difficulty with smaller objects than larger ones, Shi and Ferrari [37] use a curriculum learning [2] approach that iteratively sorts its proposed objects by size and learned weights.”
BackgroundProgressive Stochastic Learning for Noisy Labels
Han, B., Tsang, I. W.-H., Chen, L., Yu, C. P., & Fung, S.-F. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2792062 · Cited by 36
Patch-Sorted Deep Feature Learning for High Resolution SAR Image Classification
Ren, Z., Hou, B., Wen, Z., & Jiao, L. (2018) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2018.2851023 · Cited by 36
“Curriculum learning can be regarded as a continuation method to deal with minimizing nonconvex criteria [45].”
BackgroundMethodDiscriminative Neural Embedding Learning for Short-Duration Text-Independent Speaker Verification
Wang, S., Huang, Z., Qian, Y., & Yu, K. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2928128 · Cited by 35
“Strategies such as curriculum learning have also been proven very useful for both deep neural network training [50] and statistical models such as PLDA [51].”
MethodMapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning
Chen, W., Zhou, S., Pan, Z., Zheng, H., & Liu, Y. (2019) · Applied Sciences · doi:10.3390/app9204198 · Cited by 37
“To address this issue, we use the curriculum learning [43] which trains the robot with a sequence of progressively more difficult tasks.”
MethodImproving End-to-End Single-Channel Multi-Talker Speech Recognition
Zhang, W., Chang, X., Qian, Y., & Watanabe, S. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.2988423 · Cited by 47
“According to some research [9], [27], the order of the data is proven to have an influence on the model optimization, which is”
MethodSteganalysis of Digital Images Using Deep Fractal Network
Singh, B., Sur, A., & Mitra, P. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3052520 · Cited by 36
“In recent literature, this problem is usually handled by curriculum learning [42], where the model is initially trained with easy examples (embedded with”
BackgroundDeep Perceptual Preprocessing for Video Coding
Chadha, A., & Andreopoulos, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01461 · Cited by 43
“We train a deep convolutional neural network with curriculum learning [6, 43] and using multi-scale crop sizes.”
MethodLearning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection
Zhang, Z., Ge, Y., Chen, R., Tai, Y. Y., Yan, Y., Yang, J. J., Wang, C., Li, J., & Huang, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01399 · Cited by 47
“This motivates us to perform a curriculum learning procedure [3, 17], i.”
MethodControlling the Rain: from Removal to Rendering
Ni, S., Cao, X., Yue, T., & Hu, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00626 · Cited by 47
“In this paper, inspired by the curriculum learning [2], we propose a decremental training strategy that trains the samples with the largest rain intensity first, and adds a smaller rain intensity every E epochs.”
MethodLi, J., Li, Y., & Yu, T. (2021) · Case Studies in Thermal Engineering · doi:10.1016/j.csite.2021.101284 · Cited by 41
Learning Adaptive Embedding Considering Incremental Class
Yang, Y., Sun, Z.-Q., Zhu, H., Fu, Y., Zhou, Y., Xiong, H., & Yang, J. J. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3109131 · Cited by 50
“To solve this problem, we employ a learnable curriculum clustering operator, which aims to conduct clustering from easy (distinguishable) to difficult (confused) instances via curriculum learning [41], [42], [43].”
MethodCausal Inference with Knowledge Distilling and Curriculum Learning for Unbiased VQA
Pan, Y., Li, Z., Zhang, L., & Tang, J. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3487042 · Cited by 40
“Curriculum Learning (CL) [8] is a training strategy to train models from simple samples to hard samples.”
BackgroundTransformer Decoders with MultiModal Regularization for Cross-Modal Food Retrieval
Shukor, M., Couairon, G., Grechka, A., & Cord, M. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00503 · Cited by 44
“IncMargin loss: Inspired by the curriculum learning approaches [3], where the task becomes harder as the training progresses, we propose to replace the constant margin by a dynamic one.”
MethodAttentionCode: Ultra-Reliable Feedback Codes for Short-Packet Communications
Shao, Y., Özfatura, E., Perotti, A., Popović, B. M., & Gündüz, D. (2023) · IEEE Transactions on Communications · doi:10.1109/tcomm.2023.3280563 · Cited by 39
“To address this problem, we leverage curriculum learning [36] to train AttentionCode when the feedforward and feedback channel SNRs are low.”
MethodLearning to Play Guess Who? and Inventing a Grounded Language as a Consequence
Jorge, E., Kågebäck, M., Johansson, F., & Gustavsson, E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03218 · Cited by 34
“Inspired by curriculum learning described in [8], where Bengio et al.”
MethodMemory Augmented Control NetworksPreprint
Khan, A., Zhang, C., Atanasov, N., Karydis, K., Kumar, V., & Lee, D. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.05706 · Cited by 88
12-in-1: Multi-Task Vision and Language Representation LearningPreprint
Lu, J., Goswami, V., Rohrbach, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02315 · Cited by 36
“Inspired by prior multi-task literature [4] [31], we experimented with both curriculum and anti-curriculum strategies based on task difficulty.”
MethodUtilizing Longitudinal Chest X-Rays and Reports to Pre-Fill Radiology Reports
Zhu, Q., Mathai, T. S., Mukherjee, P., Peng, Y., Summers, R. M., & Lu, Z. (2023) · arXiv · doi:10.48550/arxiv.2306.08749 · Cited by 36
“One possible way to address this issue is to use active learning [23] or curriculum learning [24] methods to differentiate different types of samples and better train the machine learning models.”
Fayaz, S., Shah, S. Z. A., Din, N. M. U., Gul, N., & Assad, A. (2024) · Recent Advances in Computer Science and Communications · doi:10.2174/0126662558286875231215054324 · Cited by 55
WorldGPT: Empowering LLM as Multimodal World Model
Ge, Z., Huang, H., Zhou, M., Li, J., Wang, G., Tang, S., & Zhuang, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681488 · Cited by 116
“Therefore, we propose a gentle pretraining method, inspired by previous studies in Curriculum Learning [3].”
MethodLegal NERC with ontologies, Wikipedia and curriculum learning
Cardellino, C., Teruel, M., Alemany, L. A., & Villata, S. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2041 · Cited by 37
“The underlying intuition is that this approach reveals the global picture (Bengio et al., 2009).”
BackgroundMethodProgressive Joint Modeling in Unsupervised Single-Channel Overlapped Speech Recognition
Chen, Z., Droppo, J., Li, J., & Xiong, W. (2017) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2017.2765834 · Cited by 65
“Curriculum learning improves both the training speed and the performance of the model [16].”
BackgroundMethodSemi-Supervised Image Classification With Self-Paced Cross-Task Networks
Wu, S., Ji, Q., Wang, S., Wong, H.-S., Yu, Z., & Xu, Y. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2758522 · Cited by 37
“The basic idea behind self-paced learning [51] is to gradually learn the model from easy samples to difficult ones.”
BackgroundWang, P., & Vasconcelos, N. M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_3 · Cited by 32
“For instance, curriculum learning [4] suggests using easy samples first and hard samples latter.”
BackgroundMulti-task Learning for Left Atrial Segmentation on GE-MRI
Chen, C., Bai, W., & Rueckert, D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-12029-0_32 · Cited by 84
“Motivated by Curriculum Learning [14], we trained our network firstly with cropped images where the left atrium taking a large portion of the image and then we gradually increased the image size.”
MethodAerobatics control of flying creatures via self-regulated learning
Won, J., Park, J., & Lee, J. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275023 · Cited by 40
“The key idea of curriculum learning is to learn easy subtasks first and then increase the level of difficulty gradually [Bengio et al. 2009].”
Background
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.