Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 1,101–1,200 of 5,240
Curriculum learning with Hindsight Experience Replay for sequential object manipulation tasks
Manela, B., & Biess, A. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.10.011 · Cited by 37
“Transfer learning is used to transfer knowledge between each pair of tasks in the curriculum [15, 16].”
BackgroundLess is more: Data-efficient complex question answering over knowledge bases
Hua, Y., Li, Y.-F., Qi, G., Wu, W., Zhang, J., & Qi, D. (2020) · Journal of Web Semantics · doi:10.1016/j.websem.2020.100612 · Cited by 38
“To adaptively control the exploration-exploitation trade-o ff , we employ a curriculum learning [15] scheme to dynamically change the influence of the two reward bonuses, namely the proximity and the novelty.”
MethodmaskedFaceNet: A Progressive Semi-Supervised Masked Face Detector
Prasad, S., Li, Y., Lin, D., & Sheng, D. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00343 · Cited by 31
“The concept is somewhat resembles to curriculum learning [2].”
BackgroundKothandaraman, D., Chandra, R., & Manocha, D. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00339 · Cited by 51
“trained model and the target domain, we use curriculum learning [2, 20] in which the DNN is allowed to train on samples progressively in their increasing order of entropy of predictions.”
3D Human Pose, Shape and Texture from Low-Resolution Images and Videos
Xu, X., Chen, H., Moreno-Noguer, F., Jeni, L. A., & la Torre, F. D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3070002 · Cited by 49
“However, [53] does not explicitly consider multi-resolution training.”
BackgroundSt-Bert: Cross-Modal Language Model Pre-Training for End-to-End Spoken Language Understanding
Kim, M., Kim, G., Lee, S., & Ha, J.-W. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414558 · Cited by 33
“We adopt a curriculum pre-training [24] for better generalization and fast convergence [25].”
MethodA Physics-Driven CNN Model for Real-Time Sea Waves 3D Reconstruction
Pistellato, M., Bergamasco, F., Torsello, A., Barbariol, F., Yoo, J., Jeong, J., & Benetazzo, A. (2021) · Remote Sensing · doi:10.3390/rs13183780 · Cited by 36
“The concept of presenting to the network gradually complex examples during training is addressed in the literature as curriculum learning [48–50].”
Imbalanced Source-free Domain Adaptation
Li, X., Li, J., Zhu, L., Wang, G., & Huang, Z. H. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475487 · Cited by 54
“Inspired by curriculum learning [2] which trains the model with easy and certain samples first and gradually transitions to harder ones, we select samples whose entropy of output prediction by the classifier, i.”
MethodCurriculum Disentangled Recommendation with Noisy Multi-feedback
Chen, H., Chen, Y., Wang, X., Xie, R., Wang, R., Xia, F., & Zhu, W. (2021) · neural information processing systems · Cited by 89
Seyyedsalehi, S. Z., & Seyyedsalehi, S. A. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.05.057 · Cited by 33
“Studies have revealed that in training deep structures, directly training of all the layers simultaneously not only complicates extraction of the entire modeling potential but also that practical results are worsened by increasing the number of layers [41].”
DNN-Based PolSAR Image Classification on Noisy Labels
Ni, J. F., Xiang, D., Lin, Z., López-Martínez, C., Hu, W., & Zhang, F. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3168799 · Cited by 34
Kim, Y., Lee, K., Nam, B.-W., & Han, Y. (2023) · Journal of Computational Design and Engineering · doi:10.1093/jcde/qwad001 · Cited by 34
Glaucoma Diagnosis from Eye Fundus Images Based on Deep Morphometric Feature Estimation
Perdómo, O., Andrearczyk, V., Mériaudeau, F., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00949-6_38 · Cited by 31
Handling Missing Annotations for Semantic Segmentation with Deep ConvNets
Petit, O., Thome, N., Charnoz, A., Hostettler, A., & Soler, L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00889-5_3 · Cited by 31
“The model trained at this first stage is then used for automatically predicting labels for missing organs, using a Curriculum strategy [2] (SMILEr).”
MethodExploiting Web Images for Weakly Supervised Object Detection
Tao, Q., Yang, H., & Cai, J. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2875597 · Cited by 32
“Curriculum Learning: Our work is inspired by curriculum learning [1] scheme.”
Methodjin, W., Du, H., Zhao, B., Tian, X., Shi, B., & Yang, G. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5106265 · Cited by 122
Swarm Cooperative Navigation Using Centralized Training and Decentralized Execution
Azzam, R., Boiko, I. M., & Zweiri, Y. (2023) · Drones · doi:10.3390/drones7030193 · Cited by 47
“Curriculum learning is a training strategy in which a neural network is gradually exposed to task complexity as originally proposed in [26].”
Distillation from Heterogeneous Models for Top-K Recommendation
Kang, S., Kweon, W., Lee, D., Lian, J., Xie, X., & Yu, H. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583209 · Cited by 52
“Curriculum learning [2, 43] trains a model by gradually including data”
BackgroundMethodInfluentialUncertainty-Aware Dual-Evidential Learning for Weakly-Supervised Temporal Action Localization
Chen, M., Gao, J., & Xu, C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3308571 · Cited by 42
“Inspired by Curriculum Learning [79], we propose a method of progressive learning that leverages the snippet-level aleatoric uncertainty u a snip , which reflects the inherent difficulty of discriminating the snippet category, to help the model learn the entire action instance progressively and…”
MethodRecent Advances in Deep Learning for Channel Coding: A Survey
Matsumine, T., & Ochiai, H. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3472094 · Cited by 44
“6) Curriculum Learning: Curriculum learning, originally proposed in [129], is a training strategy that trains a machine learning model from easier data to harder data, imitating human learning [130], [131].”
Acquiring musculoskeletal skills with curriculum-based reinforcement learning
Chiappa, A. S., Tano, P., Patel, N., Ingster, A., Pouget, A., & Mathis, A. (2024) · Neuron · doi:10.1016/j.neuron.2024.09.002 · Cited by 55
Fixed-point optimization of deep neural networks with adaptive step size retraining
Shin, S., Boo, Y., & Sung, W. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952347 · Cited by 37
“The curriculum learning is a training strategy to move the goal from an easy level to more complex one gradually [13].”
BackgroundResults of the WMT17 Neural MT Training Task
Bojar, O., Helcl, J., Kocmi, T., Libovický, J., & Musil, T. (2017) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w17-4757 · Cited by 32
“The basic idea behind this technique (Bengio et al., 2009) is inspired by the fact that humans learn more easily when examples are presented in an ordering from trivial to complex ones.”
BackgroundPlaygol: Learning Programs Through Play
Cropper, A. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/841 · Cited by 30
“Curriculum learning [ Bengio et al. , 2009 ] is a similar idea but requires an ordering over the given tasks.”
BackgroundAdvancing Acoustic-to-Word CTC Model With Attention and Mixed-Units
Das, A., Li, J., Ye, G., Zhao, R., & Gong, Y. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2933325 · Cited by 27
“curriculum learning [38], Nesterov momentum-based stochastic gradient descent, dropout, and low rank matrix factorization [39].”
BackgroundCrowdsourcing the Perception of Machine Teaching
Hong, J., Lee, K., Xu, J., & Kacorri, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376428 · Cited by 30
“[35] observed that one of the most prominent teaching strategies for a binary classification task among non-experts, called the extreme strategy, is consistent with the “curriculum learning” principle [8,40], where participants start with the most extreme examples and continue with those closer to the decision boundary…”
ResultEl-Bouri, R., Eyre, D. W., Watkinson, P., Zhu, T., & Clifton, D. A. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2990309 · Cited by 30
“network parameters) is more likely to be convex [5].”
BackgroundMethodInfluentialImproving Monocular Depth Estimation by Leveraging Structural Awareness and Complementary Datasets
Chen, T., An, S., Zhang, Y., Ma, C., Wang, H., Guo, X., & Zheng, W. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58568-6_6 · Cited by 40
“When training on mixed datasets, curriculum learning [3] is needed to avoid the local minimum problem by training the model on easier datasets first.”
BackgroundMethodDiscriminative Neural Clustering for Speaker Diarisation
Li, Q., Kreyssig, F., Zhang, C., & Woodland, P. C. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383617 · Cited by 47
“A curriculum learning (CL) [39, 40] approach, i.”
Chang, E., Yeh, H.-S., & Demberg, V. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.61 · Cited by 56
Jiménez-Sánchez, A., Mateus, D., Kirchhoff, S., Kirchhoff, C., Biberthaler, P., Navab, N., Ballester, M. Á. G., & Piella, G. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102273 · Cited by 45
“AntiCL leading to a better performance than the baseline is a behaviour also reported in [24].”
BackgroundResultInfluentialDeep Reinforcement Learning for Humanoid Robot Behaviors
Muzio, A. F. V., Máximo, M. R. O. A., & Yoneyama, T. (2022) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-022-01619-y · Cited by 30
“To successfully learn this task, we employed curriculum learning [9].”
MethodGaitMPL: Gait Recognition With Memory-Augmented Progressive Learning
Dou, H., Zhang, P., Zhao, Y., Dong, L., Qin, Z., & Li, X. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3164543 · Cited by 40
“Existing progressive learning can mainly be divided into two categories: curriculum learning [48] and hard example mining (HEM) [49], which represent the “easier first” and the “harder first” learning order, respectively.”
BackgroundEfficient Join Order Selection Learning with Graph-based Representation
Chen, J., Ye, G., Zhao, Y. L., Liu, S., Deng, L., Chen, X., Zhou, R., & Zheng, K. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539303 · Cited by 43
“Curriculum learning refers to a learning strategy that learns from easier data to more difficult data [2, 5, 8, 20, 24].”
BackgroundSavelonas, M. A., Veinidis, C. N., & Bartsokas, T. (2022) · Remote Sensing · doi:10.3390/rs14236017 · Cited by 42
EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones
Wang, Y., Yang, Y., Lü, R., Liu, T., Zhong, Z., Song, S., & Huang, G. M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00538 · Cited by 58
“Typically, a pre-defined [8, 27, 28, 29] or automatically-learned [25, 30, 31, 11, 32, 16, 14, 33, 34, 12, 35] difficulty measurer is deployed to differentiate between easier and harder samples, while a scheduler [8, 11, 26, 9] is defined to determine when and how to introduce harder training data.”
BackgroundInfluentialSizeNet: Weakly Supervised Learning of Visual Size and Fit in Fashion Images
Karessli, N., Guigourès, R., & Shirvany, R. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00046 · Cited by 34
“Most approaches using a teacher-student learning strategy [21, 23, 22] derive the importance of the samples from the teacher classifier.”
BackgroundMethodInfluentialVisualizing and Understanding Curriculum Learning for Long Short-Term Memory NetworksPreprint
Cirik, V., Hovy, E. H., & Morency, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06204 · Cited by 94
“The first algorithm proposed by Bengio et al. (2009), which we refer as one-pass curriculum, creates disjoint sets of training examples ordered by the complexity and used separately during training.”
BackgroundMethodInfluentialC-MIL: Continuation Multiple Instance Learning for Weakly Supervised Object DetectionPreprint
Wan, F., Liu, C., Ke, W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05647 · Cited by 30
“In machine learning, curriculum learning [5] was inspired by this principle to define a sequence of gradually increasing difficulty training tasks (or training distributions) which converge to the task of interest.”
BackgroundKrishnan, S., Boroujerdian, B., Fu, W., Faust, A., & Reddi, V. J. (2019) · arXiv (Cornell University) · Cited by 33
Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend PredictionPreprint
Hu, Z., Liu, W., Bian, J., Liu, X., & Liu, T. (2017) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.1712.02136 · Cited by 95
“Curriculum Learning [4] is a learning mode that can imitate such a learning process.”
BackgroundAutomated curricula through setter-solver interactionsPreprint
Racanière, S., Lampinen, A. K., Santoro, A., Reichert, D., Firoiu, V., & Lillicrap, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12892 · Cited by 64
“When uniform sampling is ineffective, hand-engineered curricula may work (Elman, 1993; Bengio et al., 2009; Zaremba & Sutskever, 2014; Graves et al., 2016).”
BackgroundCurricularFace: Adaptive Curriculum Learning Loss for Deep Face RecognitionPreprint
Huang, Y., Wang, Y., Tai, Y. Y., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00288 · Cited by 30
“Motivated by the nature of human learning that easy cases are learned first and then come the hard ones [2], our CurricularFace incorporates the idea of Curriculum Learning (CL) into face recognition in an adaptive manner, which differs from the traditional CL in two aspects.”
BackgroundMethodLearning to select data for transfer learning with Bayesian Optimization
Ruder, S., & Plank, B. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1038 · Cited by 151
“Very recently interest emerged in curriculum learning (Bengio et al., 2009).”
BackgroundAn End-To-End Framework For Pose Estimation Of Occluded Pedestrians
Das, S. K., Kishore, P. S. R., & Bhattacharya, U. (2020) · Proceedings - International Conference on Image Processing · doi:10.1109/icip40778.2020.9191147 · Cited by 30
“We employ Curriculum Learning [20] in the Mask and Predict strategy by gradually increasing the masking percentage with training.”
MethodBoosting adversarial robustness via self-paced adversarial training
He, L., Ai, Q., Yang, X., Ren, Y., Wang, Q., & Xu, Z. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.08.063 · Cited by 29
SHAPE: A Sample-Adaptive Hierarchical Prediction Network for Medication Recommendation
Liu, S., Wang, X., Du, J., Hou, Y., Zhao, X., Xu, H., Wang, H., Xiang, Y., & Tang, B. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3320139 · Cited by 41
Tasnim, U., Islam, R., Desai, K., & Quarles, J. P. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3372122 · Cited by 39
“This approach ensures that the model is better equipped to handle the nuances of the dataset( I n 1 ) from the onset [3, 54].”
BackgroundMethodYu, T., Guo, C., Wang, L., Xiang, S., & Pan, C. (2018) · IEEE Signal Processing Letters · doi:10.1109/lsp.2018.2843295 · Cited by 24
“SPL inherits from curriculum learning (CL) [7], in which a curriculum determines a sequence of training samples that are ranked in ascending order of learning difficulty.”
BackgroundA Multistage Training Framework for Acoustic-to-Word Model
Yu, C., Zhang, C., Weng, C., Cui, J., & Yu, D. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1452 · Cited by 24
“The curriculum training [19] has been successfully applied for many difficult optimization problems where a challenging task is addressed by starting from learning easier subtasks.”
BackgroundLearning to Pour using Deep Deterministic Policy Gradients
Do, C., Gordillo, C., & Burgard, W. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593654 · Cited by 29
“The initial position of the cups is fixed, but techniques like curriculum learning [3] could be used to progressively explore more challenging scenarios.”
BackgroundFeng, J., Tao, C., Wu, W., Feng, Y., Zhao, D., & Yan, R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1370 · Cited by 30
“Previous studies including curriculum learning (CL) (Bengio et al., 2009) and self-paced learning (SPL) (Jiang et al., 2014, 2015) tackle the problem with heuristics, such as ordering data from easy instances to hard ones (Spitkovsky et al., 2010; Tsvetkov et al., 2016) and retaining training…”
BackgroundEfficient Diverse Ensemble for Discriminative Co-tracking
Meshgi, K., Oba, S., & Ishii, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00506 · Cited by 28
“Starting from easiest examples (Curriculum learning) [6], pruning adversarial examples [35], excluding misclassified samples from next rounds of training [51], sorting samples by their training value [30] are some of the proposed approaches in the literature.”
Density-Aware Curriculum Learning for Crowd Counting
Wang, Q., Lin, W., Gao, J., & Li, X. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3033428 · Cited by 35
“CL is a learning strategy formalized by Bengio et al. [27].”
BackgroundMe-Momentum: Extracting Hard Confident Examples from Noisily Labeled Data
Bai, Y., & Liu, T. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00918 · Cited by 51
“It has also been widely studied in the traditional classification problem that hard examples are essential to train accurate classifiers [42, 3, 12, 11].”
BackgroundClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis
Shen, I., & Chen, B. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3084944 · Cited by 33
“Inspired by the curriculum learning training strategy used in [63], [64], we categorized the entire training process of our generative model into”
MethodQiu, B., van der Wel, H., Kraeima, J., Glas, H. H., Guo, J., Borra, R., Witjes, M. J. H., & van Ooijen, P. M. A. (2021) · Journal of Personalized Medicine · doi:10.3390/jpm11060560 · Cited by 35
“…accurate mandible segmentation algorithm to overcome the inaccurate prediction for 3D VSP in CBCT. Motivated by the concept of curriculum learning [12], we propose a novel mandible segmentation approach based on a coarse-to-fine learning framework to solve the aforementioned challenges in mandible…”
BackgroundMethodInfluentialEßer, J., Bach, N., Jestel, C., Urbann, O., & Kerner, S. (2022) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2022.3207664 · Cited by 47
“77 JUNE 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE CURRICULUM LEARNING In the context of RL, curriculum learning [13] provides a framework for increasing sample efficiency through task structuring, where the policy for a complex task is learned by solving simpler tasks with gradually increasing levels of difficulty [see…”
BackgroundOnline Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries
Bang, J., Koh, H., Park, S., Song, H., Ha, J.-W., & Choi, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00906 · Cited by 81
“As argued in curriculum learning literature [6, 13], easy or clean examples are favored in the early stage of training of a model.”
BackgroundSelf-Paced Weight Consolidation for Continual Learning
Cong, W. W., Cong, Y., Sun, G., Liu, Y., & Dong, J. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3304567 · Cited by 39
“Bengio et al. propose a new learning paradigm called curriculum learning [25], which can gradually learn from simple and universal to difficult and complex samples.”
BackgroundCASED: Curriculum Adaptive Sampling for Extreme Data Imbalance
Jesson, A., Guizard, N., Ghalehjegh, S. H., Goblot, D., Soudan, F., & Chapados, N. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66179-7_73 · Cited by 59
“In the limit, as τ goes to infinity, pτ (xi) converges to a uniform distribution over xi, which makes CASED a valid curriculum [4].”
BackgroundMethodInfluentialBaby Steps: How “Less is More” in unsupervised dependency parsing
Spitkovsky, V. I., Alshawi, H., & Jurafsky, D. S. (2009) · Cited by 33
Etalumis: bringing probabilistic programming to scientific simulators at scale
Baydin, A. G., Shao, L., Bhimji, W., Heinrich, L., Meadows, L., Liu, J., Munk, A., Naderiparizi, S., Gram-Hansen, B., Louppe, G., et al. (2019) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3295500.3356180 · Cited by 74
“In multi-bucketing [10, 20, 40], traces are grouped into several buckets based on lengths, and every global minibatch is solely taken from a randomly picked bucket for every iteration.”
BackgroundDynamic R-CNN: Towards High Quality Object Detection via Dynamic TrainingPreprint
Zhang, H., Chang, H., Ma, B., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.06002 · Cited by 29
“Besides, Curriculum Learning [1] and Self-paced Learning [20] focus on improving the training order of the examples.”
BackgroundParameter Sharing is Surprisingly Useful for Multi-Agent Deep Reinforcement Learning.Preprint
Terry, J. K., Grammel, N., Hari, A., Santos, L., Black, B., & Manocha, D. (2020) · arXiv (Cornell University) · Cited by 35
Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP
Bansal, T., Gunasekaran, K. P., Wang, T., Munkhdalai, T., & McCallum, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.469 · Cited by 32
“(Bengio et al., 2009; Graves et al., 2017) instead posits that choosing the order in which instances are presented, with gradually increasing complexity, can enable faster learning and better generalization.”
Intra- and Inter-Modal Curriculum for Multimodal Learning
Zhou, Y., Wang, X. Y., Chen, H., Duan, X., & Zhu, W. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612468 · Cited by 47
“Bengio et al. [7] first give its formal definition and propose a simple method named Baby Step [65].”
BackgroundMethodScalable and Effective Generative Information Retrieval
Zeng, H., Luo, C., Jin, B., Sarwar, S. M., Wei, T., & Zamani, H. (2024) · Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589334.3645477 · Cited by 32
“To better learn representations aligned with the left-to-right decoding characteristic of the beam search, we draw inspiration from curriculum learning [2, 25, 27, 49] and implement a progressive training strategy.”
MethodA systematic literature review on incomplete multimodal learning: techniques and challenges
Zhan, Y., Yang, R., You, J., Huang, M., Liu, W., & Liu, X. (2025) · Systems Science & Control Engineering · doi:10.1080/21642583.2025.2467083 · Cited by 72
“Curriculum learning progressively escalates the complexity of the training data (Bengio et al., 2009; X. Wang et al., 2021), enabling the model to learn to handle missing modalities from simple to complex tasks, as shown in Figure 17.”
Active contextual policy search
Fabisch, A., & Metzen, J. H. (2014) · Journal of Machine Learning Research · doi:10.5555/2627435.2697072 · Cited by 38
CASSL: Curriculum Accelerated Self-Supervised Learning
Murali, A., Pinto, L., Gandhi, D., & Gupta, A. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8463147 · Cited by 35
“Under the name of Curriculum Learning (CL) [10], the core idea is to learn easier aspects of the problem earlier while gradually increasing the difficulty.”
BackgroundMulti-Modal Curriculum Learning over Graphs
Gong, C., Yang, J. J., & Tao, D. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3322122 · Cited by 27
“[4], which mimics the cognitive process of humans and favors a learning algorithm to follow the logical learning sequence from simple examples tomore difficult ones.”
BackgroundGaining Extra Supervision via Multi-task learning for Multi-Modal Video Question Answering
Kim, J., Ma, M., Kim, K., Kim, S.-J., & Yoo, C. D. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852087 · Cited by 29
“Motivated by curriculum learning [23], multi-task ratio scheduling attempts to learn easier task earlier to set inductive bias at the beginning of the training.”
BackgroundEnd-to-end Deep Reinforcement Learning for Multi-agent Collaborative Exploration
Chen, Z., Subagdja, B., & Tan, A. (2019) · IEEE International Conference on Agents (ICA) · doi:10.1109/agents.2019.8929192 · Cited by 35
“We define a curriculum as training distribution Qλ based on [14] which can be formulated as follows, Qλ(z) ∝Wλ(z)P (z) ∀z.”
MethodA Multi-Target SNR-Progressive Learning Approach to Regression Based Speech Enhancement
Tu, Y.-H., Du, J., Gao, T., & Lee, C. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.2996503 · Cited by 28
“Similar to ensemble learning, another notable machine learning strategy is the curriculum learning [38] originated from cog-”
BackgroundReconstructing Reflection Maps Using a Stacked-CNN for Mixed Reality Rendering
Chalmers, A., Zhao, J., Medeiros, D., et al. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3001917 · Cited by 28
“This follows the general idea of curriculum learning [28], [29], which is the process of solving easier problems first and gradually increasing the difficulty.”
BackgroundSuppressing Mislabeled Data via Grouping and Self-attention
Peng, X., Wang, K., Zeng, Z., Li, Q., Yang, J., & Qiao, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_46 · Cited by 40
“Some others resort to curriculum learning [4] by either designing a step-wise easy-to-hard strategy for training [10] or introducing an extra MentorNet [12] for sample weighting.”
BackgroundDynamic Curriculum Learning for Low-Resource Neural Machine Translation
Xu, C., Hu, B., Jiang, Y., Feng, K., Wang, Z., Huang, S., Ju, Q., Xiao, T., & Zhu, J. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.352 · Cited by 38
“Curriculum learning (Bengio et al., 2009) is motivated by the learning strategy of biological organisms which orders the training samples in an easy-to-hard manner (Elman, 1993).”
InfluentialSledge, I. J., Emigh, M. S., King, J. L., Woods, D., Cobb, J. T., & Príncipe, J. C. (2022) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2022.3152863 · Cited by 37
“Another benefit of this cost function is that it yields a blend of self-paced-learning [109] and curriculum-learning [110] regimes.”
BackgroundKnowledge distillation via instance-level sequence learning
Zhao, H., Sun, X., Dong, J., Dong, Z., & Li, Q. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107519 · Cited by 36
“From the perspective of curriculum learning [29], the”
BackgroundXue, H., Hein, B., Bakr, M. H., Schildbach, G., Abel, B., & Rueckert, E. (2022) · Applied Sciences · doi:10.3390/app12063153 · Cited by 36
“The term Curriculum Learning was first proposed by [54].”
BackgroundReward modeling for mitigating toxicity in transformer-based language models
Faal, F., Schmitt, K., & Yu, J. Y. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03944-z · Cited by 60
“ing applications, recent findings indicate that an anticurriculum schedule strategy provides better results compared to a fully joint sampling strategy [36], [37].”
ResultQLP: Deep Q-Learning for Pruning Deep Neural Networks
Camci, E., Gupta, M., Wu, M., & Lin, J. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3167951 · Cited by 32
“Drawing inspiration from curriculum learning [47], our 325 agent breaks down the overall pruning process into 326 tasks (stages) with increasing complexity, as depicted in Fig.”
MethodBio-Inspired Rhythmic Locomotion for Quadruped Robots
Sheng, J., Chen, Y., Fang, X., Zhang, W., Song, R., Zheng, Y., & Li, Y. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3177289 · Cited by 30
Share with Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency
Monnier, T., Fisher, M., Efros, A. A., & Aubry, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19769-7_17 · Cited by 42
Rank-based Decomposable Losses in Machine Learning: A Survey
Hu, S., Wang, X., & Lyu, S. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3296062 · Cited by 44
Towards End-to-end Text Spotting with Convolutional Recurrent Neural NetworksPreprint
Li, H., Wang, P., & Shen, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.03985 · Cited by 28
“Curriculum Learning In order to improve generalization and accelerate the convergence speed, we design a curriculum learning [1] paradigm to train the model from gradually more complex data.”
MethodMix&Match - Agent Curricula for Reinforcement LearningPreprint
Czarnecki, W. M., Jayakumar, S. M., Jaderberg, M., Hasenclever, L., Teh, Y. W., Osindero, S., Heess, N., & Pascanu, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01780 · Cited by 82
“Curriculum learning is a powerful instrument in the deep learning toolbox (e.g. Bengio et al., 2009; Graves et al., 2017).”
BackgroundMethodGeneral Greedy De-Bias Learning
Han, X., Wang, S., Su, C., Huang, Q., & Tian, Q. C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3240337 · Cited by 37
“Furthermore, inspired by Curriculum Learning [33], −LCE(f(X), σ̂) can be regarded as a soft difficulty measurement for curriculum sample selection function.”
BackgroundCurriculum-Based Augmented Fourier Domain Adaptation for Robust Medical Image Segmentation
Wang, A., Islam, M., Xu, M., & Ren, H. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3295600 · Cited by 40
“Recently, curriculum learning [25], a training scheme that aims to let the model learn from easier to more complex samples or tasks, has been captivating increasing attention in the field of computer vision.”
BackgroundFeng, X., Feng, X., Qin, B., & Liu, T. (2023) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2023.3314642 · Cited by 49
“We adopt a One-Pass scheduler with a linear pace [3] to progressively train the model in an easy-to-hard order.”
Continual Test-time Domain Adaptation via Dynamic Sample Selection
Wang, Y., Hong, J., Cheraghian, A., Rahman, S., Ahmedt‐Aristizabal, D., Petersson, L., & Harandi, M. T. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00172 · Cited by 72
“To achieve this objective, we employ adaptive pseudo-labelling from recent advances in semi-supervised learning [26, 30], a learning approach that involves continuous monitoring of the model’s learning progress in accordance with its learning status [1].”
MethodSensing the diversity of rumors: Rumor detection with hierarchical prototype contrastive learning
Zheng, P., Dou, Y., & Yan, Y. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103832 · Cited by 32
Tackling Noisy Clients in Federated Learning with End-to-end Label Correction
Jiang, X., Sun, S., Li, J., Xue, J., Li, R., Wu, Z., Xu, G., Wang, Y., & Liu, M. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679550 · Cited by 55
Exploiting Target Data to Learn Deep Convolutional Networks for Scene-Adapted Human Detection
Wu, S., Wang, S., Laganière, R., Liu, C., Wong, H., & Xu, Y. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2779271 · Cited by 26
“The basic idea behind self-paced learning [47] is to gradually train the model from easy samples to difficult ones.”
BackgroundGradual Machine Learning for Entity Resolution
Hou, B., Chen, Q., Shen, J., Liu, X., Zhong, P., Wang, Y., Chen, Z., & Li, Z. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3314121 · Cited by 28
“We note that there already exist many learning paradigms for a variety of classification tasks, including transfer learning [36], lifelong learning [13], curriculum learning [5], self-paced learning [28] and self-training learning [31] to name a few.”
BackgroundLarge Factor Image Super-Resolution With Cascaded Convolutional Neural Networks
Zhang, D., Shao, J., Liang, Z., Gao, L., & Shen, H. T. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3008041 · Cited by 28
“Bengio et al. [60] showed that it is easier for humans and animals to understand or learn if the learning samples present in a meaningful order which illustrates gradually more concepts, andgraduallymorecomplexones.”
BackgroundDeep Unsupervised Self-Evolutionary Hashing for Image Retrieval
Zhang, H., Gu, Y., Yao, Y., Zhang, Z., Liu, L., Zhang, J., & Shao, L. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3025000 · Cited by 27
“This strategy constrains the data in an organized training, which does not need predefined information such as the number of categories, and also can help to avoid falling into local minimum [48].”
BackgroundMADRaS : Multi Agent Driving Simulator
Santara, A., Rudra, S., Buridi, S. A., Kaushik, M., Naik, A., Kaul, B., & Ravindran, B. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12531 · Cited by 40
“Curriculum learning (Bengio et al., 2009) has been shown to be effective in reducing the sample complexity in such problems.”
BackgroundSuggestive Annotation of Brain Tumour Images with Gradient-Guided Sampling
Dai, C., Wang, S., Mo, Y., Zhou, K., Angelini, E. D., Guo, Y., & Bai, W. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59719-1_16 · Cited by 45
“As revealed in [16,17], less informative samples are more important at the early stage of model training, while harder samples are more important at the later stage.”
Background
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.