Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 2,301–2,400 of 5,240
Tied Spatial Transformer Networks for Digit Recognition
Cîrstea, B.-I., & Likforman-Sulem, L. (2016) · International Conference on Frontiers in Handwriting Recognition · doi:10.1109/icfhr.2016.0102 · Cited by 8
“Another manner to interpret the results is in relationship to curriculum learning [21].”
BackgroundMulti-task classification with sequential instances and tasks
Xu, W. H., Liu, W., Chi, H., et al. (2018) · Signal Processing Image Communication · doi:10.1016/j.image.2018.02.013 · Cited by 8
Curriculum Learning Strategies for Hindi-English Code-Mixed Sentiment Analysis
Dahiya, A., Battan, N., Shrivastava, M., & Sharma, D. M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-56150-5_9 · Cited by 9
Learning in Growing Robots: Knowledge Transfer from Tadpole to Frog Robot
Zhu, Y., Rossiter, J. M., & Häuser, H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-24741-6_42 · Cited by 8
“This concept of reusing and transferring knowledge has been explored previously by shaping [2], lifelong learning [3], curriculum learning [4], and transfer learning [5].”
BackgroundDisentangled Representations in Neural ModelsPreprint
Whitney, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02383 · Cited by 16
“Various experiments done by (Bengio et al. 2009, Zaremba and Sutskever (2014), Reed and Freitas (2015)), among many others, have shown using curricula with increasing complexity to be extremely effective.”
MethodFast Gated Neural Domain Adaptation: Language Model as a Case Study
Zhang, J., Wu, X., Way, A., & Liu, Q. (2016) · Dublin City University Open Access Institutional Repository (Dublin City University) · Cited by 11
Adversarial Generation of Training Examples for Vehicle License Plate Recognition.Preprint
Wang, X., You, M., & Shen, C. (2017) · arXiv (Cornell University) · Cited by 8
Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill ClimbingPreprint
Plessen, M. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10785 · Cited by 8
“2) The problematic of curriculum learning: In [33], curriculum learning (CL) is discussed as a method to speed up learning by providing the learning agent first with simpler examples before gradually increasing complexity.”
BackgroundMethodInfluentialA Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban ScenesPreprint
Zhang, Y., David, P., Foroosh, H., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.09953 · Cited by 8
“Accordingly, we propose a curriculum-style [33] domain adaptation approach.”
MethodIncremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and IntransigencePreprint
Kim, D., Bae, J.-H., Jo, Y., & Choi, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00829 · Cited by 28
“…to learn a model with both less forgetting and less intransigence, we propose a DropOut Sampling that is performed at every mini-batch of the stochastic gradient descent (SGD) process, inspired by curriculum learning (Bengio et al., 2009) and self-paced learning (Kumar et al., 2010) literature.”
MethodFrom Few to More: Large-scale Dynamic Multiagent Curriculum LearningPreprint
Wang, W., Yang, T., Liu, Y. K., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.02790 · Cited by 8
“Curriculum Learning (CL) is firstly introduced in (Bengio et al. 2009) which is defined as a Machine Learning notion to improve the performance of Supervised Learning.”
BackgroundRL-LIM: Reinforcement Learning-based Locally Interpretable ModelingPreprint
Yoon, J., Arık, S. Ö., & Pfister, T. (2019) · arXiv (Cornell University) · Cited by 10
ZPD Teaching Strategies for Deep Reinforcement Learning from DemonstrationsPreprint
Seita, D., Chan, D. M., Rao, R., Tang, C., Mandi, Z., & Canny, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12154 · Cited by 16
“Another interpretation of this matching function is as a curriculum [3] where the learner initially starts with a teacher that provides “easier” samples, before moving on to harder teachers.”
BackgroundMethodFine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine TranslationPreprint
Guo, J., Tan, X., Xu, L.-L., Qin, T., Chen, E.-H., & Liu, T.-Y. (2019) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.1911.08717 · Cited by 93
“Inspired by that, Bengio et al. (2009) propose curriculum learning, a machine learning training strategy that feeds training instances to the model from easy to hard.”
BackgroundMethodInfluentialLearning Hierarchical Teaching Policies for Cooperative AgentsPreprint
Kim, D.-K., Liu, M., Omidshafiei, S., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03216 · Cited by 8
“The field of curriculum learning (Bengio et al., 2009; Tsvetkov et al., 2016; Table 1.”
BackgroundLeveraging End-to-End Speech Recognition with Neural Architecture SearchPreprint
Baruwa, A., Abisiga, M., Gbadegesin, I., & Fakunle, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.05946 · Cited by 15
“Many difficult learning tasks can be easily accomplished by stacking layers of neurons as shown by recent research results [1], [2], [3], [4], [5], [6] compared with traditional machine learning algorithms that have been in common use in the past.”
ResultRobot Navigation with Map-Based Deep Reinforcement LearningPreprint
Chen, G., Pan, L., Chen, Y., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04349 · Cited by 8
“Curriculum learning [24] is a learning strategy in machine learning, which starts from easy instances and then gradually handles harder ones.”
BackgroundCurriculum in Gradient-Based Meta-Reinforcement LearningPreprint
Mehta, B., Deleu, T., Raparthy, S. C., Pal, C., & Paull, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.07956 · Cited by 17
“Starting with the seminal curriculum learning paper [Bengio et al., 2009], many different proposals to learn an optimal ordering over tasks has been studied.”
BackgroundMVP: Unified Motion and Visual Self-Supervised Learning for Large-Scale Robotic NavigationPreprint
Chancán, M., & Milford, M. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00667 · Cited by 10
“Reward design and curriculum learning: We use multiple levels of curriculum learning [87] to gradually encourage”
MethodBabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby StepsPreprint
Zhu, W., Hu, H., Chen, J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04625 · Cited by 8
“Curriculum Learning Since proposed in (Bengio et al., 2009), curriculum learning was successfully used in a range of tasks: training robots for goal reaching (Florensa et al., 2017), visual question answering (Mao et al., 2019), image generation (Karras et al., 2018).”
BackgroundMethodNorm-Based Curriculum Learning for Neural Machine TranslationPreprint
Liu, X., Lai, H., Wong, D. F., & Chao, L. (2020) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2006.02014 · Cited by 136
“Curriculum learning (CL), which aims to train machine learning models better and faster (Bengio et al., 2009), is gaining an intuitive appeal to both academic and industrial NMT systems.”
BackgroundEscolano, C., Costa‐jussà, M. R., Fonollosa, J. A. R., & Artetxe, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01594 · Cited by 13
Derivative Manipulation for General Example WeightingPreprint
Wang, X., Kodirov, E., Yang, H., & Robertson, N. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11233 · Cited by 16
Curriculum Manager for Source Selection in Multi-Source Domain AdaptationPreprint
Yang, L., Balaji, Y., Lim, S.-N., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.01261 · Cited by 8
“To address the MS-UDA problem, we propose an adversarial agent that learns a dynamic curriculum [4] for multiple source domains, named Curriculum Manager for Source Selection (CMSS).”
MethodGuided Curriculum Learning for Walking Over Complex TerrainPreprint
Tidd, B., Hudson, N., & Cosgun, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.03848 · Cited by 26
“Bengio et al [Bengio et al., 2009] highlight that for complex loss functions, CL can guide training towards better regions, helping find more suitable local minima.”
BackgroundSmooth activations and reproducibility in deep networksPreprint
Shamir, G. I., Lin, D., & Coviello, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09931 · Cited by 25
“Like humans, they may establish different beliefs as function of the order in which they had seen the training data (Achille et al., 2017; Bengio et al., 2009).”
BackgroundTowards Robotic Assembly by Predicting Robust, Precise and Task-oriented GraspsPreprint
Zhao, J. A., Troniak, D. M., & Kroemer, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.02462 · Cited by 13
“Curriculum learning [25] has been proven to be able to outperform learning from randomly presented training data by gradually increasing the complexity of presented data during training.”
BackgroundCircus ANYmal: A Quadruped Learning Dexterous Manipulation with Its LimbsPreprint
Shi, F., Homberger, T., Lee, J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.08811 · Cited by 8
“Curriculum learning [50] is an effective strategy to solve challenging tasks, in which the task difficulties gradually increase during the learning process.”
BackgroundPREDATOR: Registration of 3D Point Clouds with Low OverlapPreprint
Huang, S., Gojčič, Ž., Usvyatsov, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13005 · Cited by 8
“On odometryKITTI dataset, we take the curriculum learning [4] strategy to gradually learn sharper local descriptors by adjusting n p .”
MethodReweighting Augmented Samples by Minimizing the Maximal Expected LossPreprint
Yi, M., Hou, L., Shang, L., Jiang, X., Liu, Q., & Ma, Z.-M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.08933 · Cited by 23
“Curriculum learning (Bengio et al., 2009; Jiang et al., 2014) feeds first easier and then harder data into the model to accelerate training.”
BackgroundAdaptive Consistency Regularization for Semi-Supervised Transfer LearningPreprint
Abuduweili, A., Li, X., Shi, H., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.02193 · Cited by 8
“Which can be regarded as a kind of curriculum learning[1].”
BackgroundA Survey on Curriculum LearningPreprint
Wang, X., Chen, Y., & Zhu, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13166 · Cited by 8
“To design a better update strategy for λ, some works [48], [73] adopt a strategy analogous to Baby Step scheduler in Sec.”
BackgroundMethodInfluentialWang, H., Planas, R., Chandramowlishwaran, A. M., et al. (2021) · arXiv (Cornell University) · Cited by 8
GraphFormers: GNN-nested Language Models for Linked Text Representation.Preprint
Yang, J., Liu, Z., Xiao, S., Li, C., Sun, G., & Xie, X. (2021) · arXiv (Cornell University) · Cited by 7
Learning Fast Sample Re-weighting Without Reward DataPreprint
Zhang, Z., & Pfister, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03216 · Cited by 8
“This phenomenon is prominent at early training phases [51, 34] because models tend to learn simple samples before hard (mislabeled) samples [45, 3].”
BackgroundConfidence-Aware Paced-Curriculum Learning by Label Smoothing for Surgical Scene Understanding
Xu, M., Islam, M., Glocker, B., & Ren, H. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3276361 · Cited by 8
“Recently, curriculum learning [6] and self-paced learning [7] have attracted the interest of people in machine learning and computer vision due to their outstanding generalization ability.”
BackgroundDecoupled Progressive Distillation for Sequential Prediction with Interaction Dynamics
Hu, K., Li, L., Xie, Q., Liu, J., Tao, X., & Xu, G. (2023) · ACM Transactions on Information Systems · doi:10.1145/3632403 · Cited by 8
Attention-Based Value Classification Reinforcement Learning for Collision-Free Robot Navigation
Sun, C., Wu, X. L., Wang, Y., & Sun, C. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3391007 · Cited by 8
“To tackle such difficulty, a two-stage training process for the AVCAC model is proposed through the inspiration of curriculum learning paradigm [43].”
On-Device Deep Learning: Survey on Techniques Improving Energy Efficiency of DNNs
Boumendil, A., Bechkit, W., & Benatchba, K. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3430028 · Cited by 13
“Some approaches [124], [125] use such assistant models with curriculum learning [126], which consists in providing the examples in a specific order according to their difficulty.”
Statewide Visual Geolocalization in the Wild
Fervers, F., Bullinger, S., Bodensteiner, C., Arens, M., & Stiefelhagen, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72764-1_25 · Cited by 26
GRACE: GRadient-based Active Learning with Curriculum Enhancement for Multimodal Sentiment Analysis
Li, X., Ye, W., Zhang, Y., & Sun, X. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681617 · Cited by 12
LEROjD: Lidar Extended Radar-Only Object Detection
Palmer, P., Krueger, M., Schütte, S., Altendorfer, R., Adam, G., & Bertram, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73027-6_22 · Cited by 14
Xie, R., Xing, J., Zhao, Q., yang, Z., & Wang, Z. (2025) · International Journal of Aeronautical and Space Sciences · doi:10.1007/s42405-025-00895-w · Cited by 7
“Ben-gio et al. [22] explained that scheduling examples and tasks by difficulty can help improve generalizability and learning speed.”
Integrated Communication and Learned Recognizer With Customized RIS Phases and Sensing Durations
Huang, Y., Yang, J., Wen, C.-K., et al. (2025) · IEEE Transactions on Communications · doi:10.1109/tcomm.2025.3548768 · Cited by 7
“Thus, we address this problem by employing the curriculum learning strategy [46].”
Miller, B., & Ziemiański, L. (2025) · Materials · doi:10.3390/ma18071469 · Cited by 11
“This idea was first introduced into machine learning by Bengio et al. [24] in 2009, who suggested that such an approach not only accelerates the training process but also improves the quality of the local minima obtained.”
Precise length control for large language models
Butcher, B., O’Keefe, M., & Titchener, J. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100143 · Cited by 33
Hocker, D. L., Constantinople, C. M., & Savin, C. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01029-3 · Cited by 12
“…can boost learning by slowly increasing task complexity along a particular dimension (for example, gradually increasing the delay in a task requiring working memory) as a form of curriculum learning 3,4 or by using families of related tasks and harnessing shared structure, as in meta-learning 5–7 .”
Sim-to-Real Transfer of Deep Reinforcement Learning Agents for Online Coverage Path Planning
Jonnarth, A., Johansson, O., Zhao, J. W., & Felsberg, M. (2025) · IEEE Access · doi:10.1109/access.2025.3581035 · Cited by 15
“To improve the convergence in the early parts of training, we apply curriculum learning [59], which has shown to be effective in reinforcement learning [60].”
Myoassist 0.1: Myosuite for Dexterity and Agility in Bionic Humans
Tan, C. K., Wang, C., Lyu, S., Hodossy, B. K., Schumacher, P., Wilson, E. B., Caggiano, V., Kumar, V., Farina, D., Gionfrida, L., et al. (2025) · International Conference on Rehabilitation Robotics · doi:10.1109/icorr66766.2025.11063089 · Cited by 7
“This baseline policy combines a simple position MyoArm Control : The myoArm policy was trained using a heuristically designed curriculum learning setup [23].”
Fan, Z., Ghaddar, B., Wang, X., Xing, L., Zhang, Y. A., & Zhou, Z. (2025) · European Journal of Operational Research · doi:10.1016/j.ejor.2025.08.029 · Cited by 48
Hard Sample Mining: A New Paradigm of Efficient and Robust Model Training
Liu, L., Liang, Y., Yan, X., Huangfu, L., Samtani, S., Yu, Z., Zhang, Y., & Zeng, D. D. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3610948 · Cited by 14
“Inspired by the learning processing of human beings, numerous innovative strategies are proposed based on curriculum learning (CL) [132], [133].”
SVGen: Interpretable Vector Graphics Generation with Large Language Models
Wang, F., Zhao, Z., Liu, Y., Zhang, D., Gao, J., Sun, H. S., & Li, X. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755011 · Cited by 38
“Curriculum learning[6] is a training strategy that mimics the human learning process.”
Rashid, H. U., Pachalieva, A., & O'Malley, D. (2025) · Scientific Reports · doi:10.1038/s41598-026-37063-3 · Cited by 7
Complementary tasks for context-dependent deep neural network acoustic models
Bell, P., & Renals, S. J. (2015) · Interspeech (USB) · doi:10.21437/interspeech.2015-716 · Cited by 7
“Multitask learning is closely related to other methods proposed for improving neural network generalisation: for example, in curriculum learning [19] the network is trained on initially on a simple task B (in the sense of the distribution of y t”
MethodIncremental learning of latent structural SVM for weakly supervised image classification
Durand, T., Thome, N., Cord, M., & Picard, D. (2014) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2014.7025862 · Cited by 10
“In [15, 18, 19], different iterative methods based on curriculum learning [22] are introduced to find a better optimum.”
MethodScale-Invariant Multi-Oriented Text Detection in Wild Scene Image
Dasgupta, K., Das, S. K., & Bhattacharya, U. (2020) · Proceedings - International Conference on Image Processing · doi:10.1109/icip40778.2020.9191102 · Cited by 8
“Also, we use curriculum learning [24] by manually arranging images from simpler to difficult ones.”
MethodTowards Hybrid Model for Automatic Text Summarization
Pei, J., Hantach, R., Abbès, S. B., & Calvez, P. (2020) · International Conference on Machine Learning and Applications · doi:10.1109/icmla51294.2020.00160 · Cited by 13
“A new method [9], has been presented for sentence extraction using curriculum learning [2].”
MethodWu, Z. B., Yao, Z. Q., & Lu, M. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3142274 · Cited by 7
Towards designing a generic and comprehensive deep reinforcement learning framework
Nguyen, N. D., Nguyen, T. T., Pham, N. T., Nguyen, V., Nguyen, D. T., Nguyen, T. T., Lim, C. P., Johnstone, M., Bhatti, A., Creighton, D., et al. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03550-z · Cited by 9
“Andrychowicz et al. [125] proposed Hindsight Experience Replay to implicitly simulate curriculum learning [126] by creating imagined trajectories in experience replay with positive rewards.”
BackgroundTrajectory Prediction from Hierarchical Perspective
Qian, T., Xu, Y., Zhang, Z., & Wang, F. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548092 · Cited by 8
“Researchers design difficulty functions to estimate the overall difficulty of samples, then adopt curriculum learning [3] to enable models to gradually proceed from easy samples to more complex ones in training.”
BackgroundOne Size Does NOT Fit All: Data-Adaptive Adversarial Training
Yang, S., & Xu, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_5 · Cited by 17
“We attribute this benefit to the dynamic ρ , for it can provide the model with a learning environment as Curriculum Learning [2], with which the robustness can be smoothly and better improved.”
MethodOverfitting-avoiding goal-guided exploration for hard-exploration multi-goal reinforcement learning
Han, C., Peng, Z., Liu, Y., Tang, J., Yu, Y., & Zhou, Z. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.01.016 · Cited by 8
Learning Nonprehensile Dynamic Manipulation: Sim2real Vision-Based Policy With a Surgical Robot
Gondokaryono, R., Haiderbhai, M., Suryadevara, S. A., & Kahrs, L. A. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3312038 · Cited by 8
“To combat this we utilize curriculum learning [27], and start with small ranges that incrementally increase as the next lesson is triggered.”
MethodMulti-Level Curriculum Learning for Multi-Turn Dialogue Generation
Chen, G., Zhan, R., Wong, D. F., & Chao, L. S. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3322583 · Cited by 11
“Curriculum learning [7] is an effective training strategy that accelerates the training process and improves model performance in lots of tasks [28], [29], [30], [31], [32], [33].”
BackgroundOmeliyanenko, J., Zehe, A., Hotho, A., & Schlör, D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47240-4_33 · Cited by 9
Prototypical Mixing and Retrieval-based Refinement for Label Noise-resistant Image Retrieval
Yang, X., Wang, H., Sun, J., Zhang, S., Chen, C., Hua, X., & Luo, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01032 · Cited by 10
“To address the problems, we provide a retrieval-based refinement module, which re-trieves comparable samples for each prototype with curriculum learning [2] for precise and balanced refinement.”
BackgroundMethodBeyond Words : Harnessing GPT-2 to Continue Stories with Imagination
Bandyopadhyay, A., Chakraborty, P., Debdas, S., Patra, M., Mohapatra, S., & Guha, D. (2023) · IEEE Silchar Subsection Conference (SILCON) · doi:10.1109/silcon59133.2023.10404413 · Cited by 9
Revisiting the Effective Number Theory for Imbalanced Learning
Wu, O., & Li, M. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3367949 · Cited by 7
Williams, D. S., Martini, D. D., Gadd, M., & Newman, P. M. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3401020 · Cited by 9
“The motivation for this comes from curriculum learning [49], where a model’s performance is improved by using a training procedure in which the difficulty of training examples increases during training.”
BackgroundMarkantonakis, K., Meister, J. A., Gurulian, I., Shepherd, C., Akram, R. N., Ghazalah, S. A., Kasi, M. K., Sauveron, D., & Hancke, G. P. (2024) · IEEE Access · doi:10.1109/access.2024.3479729 · Cited by 13
“There are many successful examples of deep learning techniques being applied to classification problems in academic literature [44].”
BackgroundActive learning of data-assimilation closures using graph neural networks
Quattromini, M., Bucci, M. A., Cherubini, S., & Semeraro, O. (2025) · Theoretical and Computational Fluid Dynamics · doi:10.1007/s00162-025-00737-1 · Cited by 10
“The first strategy is inspired by the curriculum learning framework [7], where each selected case is added to the ongoing training.”
Mining Relations from Unstructured Content
Lourentzou, I., Alba, A., Coden, A. R., Gentile, A. L., Gruhl, D., & Welch, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93037-4_29 · Cited by 8
“We used curriculum learning (CL) strategies [6], where the order of the data is decided in advance - before starting the learning process using several text based criteria.”
MethodUnderstanding ecommerce clickstreams: a tale of two states
Sheil, H., RANA, O. F., & Reilly, R. B. (2018) · ORCA Online Research @Cardiff (Cardiff University) · Cited by 7
Developmental Reinforcement Learning through Sensorimotor Space Enlargement
Zimmer, M., Boniface, Y., & Dutech, A. (2018) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2018.8761021 · Cited by 7
“Like curriculum learning [4], we suppose that learning with gradual difficulty is helpful to explore the search space in complex continuous environments.”
BackgroundCao, L., He, S., Liu, K., & Zhao, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1367 · Cited by 10
“From the point of performances on each decoder, the effectiveness of multi-pass decoder could be explained from curriculum learning (Bengio et al., 2009).”
BackgroundModel Adaptation with Synthetic and Real Data for Semantic Dense Foggy Scene Understanding
Sakaridis, C., Dai, D., Hecker, S., et al. (2018) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000305722 · Cited by 7
Learning to Inpaint by Progressively Growing the Mask Regions
Hedjazi, M. A., & Genç, Y. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00562 · Cited by 6
“Another attempt to stabilize the training of GANs is to employ a curriculum learning (CL) approach [21].”
BackgroundEnd-to-End Overlapped Speech Detection and Speaker Counting with Raw Waveform
Zhang, W., Sun, M., Wang, L., & Qian, Y. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003962 · Cited by 8
“However, some works [14, 15] have proven the order of training data can influence the training process, especially when the data are sorted from “easiest” to “hardest”, which is called curriculum learning.”
BackgroundCityLearn: Diverse Real-World Environments for Sample-Efficient Navigation Policy Learning
Chancán, M., & Milford, M. J. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197336 · Cited by 8
“Reward design and curriculum learning: We use 7 levels of curriculum learning [14] to encourage the agent to”
BackgroundMethodModular Deep Reinforcement Learning for Emergent Locomotion on a Six-Legged Robot
Schilling, M., Konen, K., & Korthals, T. (2020) · IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) · doi:10.1109/biorob49111.2020.9224332 · Cited by 8
“Training on a simpler terrain beforehand seems to be beneficial and is similar to a form of curriculum learning [36].”
BackgroundAdaptive Graph Learning for Semi-supervised Self-paced Classification
Chen, L., & Lu, J. (2021) · Neural Processing Letters · doi:10.1007/s11063-021-10453-6 · Cited by 7
“[28] proposed a new learning concept called curriculum learning.”
BackgroundCross-Domain Error Minimization for Unsupervised Domain Adaptation
Du, Y., Chen, Y., Cui, F., Zhang, X., & Wang, C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-73197-7_29 · Cited by 9
“Moreover, we propose to use curriculum learning [2] based strategy to select target samples with high prediction confidence during training.”
MethodLearning Error-Driven Curriculum for Crowd Counting
Li, W., Cao, Z., Wang, Q., et al. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413068 · Cited by 7
“Inspired by the recent success of curriculum learning [21], [22], we tackle data imbalance by learning a curriculum.”
BackgroundMethodKosaian, J., Phanishayee, A., Philipose, M., Dey, D., & Vinayak, R. (2021) · International Conference on Machine Learning · Cited by 31
Learning from the Web: Webly Supervised Meta-Learning for Masked Face Recognition
Zheng, W., Yan, L., Wang, F., & Gou, C. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00486 · Cited by 9
“Hence, in Stage II, we adopt a curriculum learning-based strategy [1, 39, 14] in training.”
BackgroundMethodZhu, W., Liu, J., Xu, J., Chen, Y., & Zhang, Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-84186-7_24 · Cited by 7
“2 Curriculum Learning Curriculum learning (Bengio et al., 2009) is a particular learning paradigm in machine learning, which starts from easy instances and then gradually handles harder ones.”
BackgroundMethodInfluentialEmploying deep learning for detection of gravitational waves from compact binary coalescences
Verma, C. K., Reza, A., Krishnaswamy, D., Caudill, S., & Gaur, G. (2022) · AIP conference proceedings · doi:10.1063/5.0108682 · Cited by 9
“At this level, in order to achieve higher classification accuracy, we employed the curriculum learning [9] method for the training.”
MethodAn Improved Deep Model for Knowledge Tracing and Question-Difficulty Discovery
Dai, H., Zhang, Y., Yun, Y., & Shang, X. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89363-7_28 · Cited by 8
On the Role of Corpus Ordering in Language Modeling
Agrawal, A., Singh, S., Schneider, L. R., & Samuels, M. (2021) · Workshop on Simple and Efficient Natural Language Processing (SustaiNLP) · doi:10.18653/v1/2021.sustainlp-1.15 · Cited by 10
“…focus is on studying the role of corpus ordering in language modeling, and although a handful of works explored this idea via length-based metrics (Bengio et al., 2009; Graves et al., 2017), they did not integrate document-level measures of complexity, nor did they study this in the context of…”
BackgroundMethodInfluentialLearning to Design and Construct Bridge without Blueprint
Li, Y., Kong, T., Li, L., Li, Y., & Wu, Y. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636280 · Cited by 14
“Curriculum learning [22] is a common practice in many robotic learning tasks [23], [24] by always training the agent with tasks of moderate difficulty.”
Backgroundde Kleijn, R., Sen, D., & Kachergis, G. (2022) · Topics in Cognitive Science · doi:10.1111/tops.12595 · Cited by 7
“Critical periods in human development—where new skills are easy to acquire only within a specific time window, not earlier or later—have been hypothesized and investigated in a variety of domains (for a review see Bornstein, 2014).”
BackgroundFu, P., Zhang, D., Yin, F., & Tang, H. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-06980-5 · Cited by 7
“To realize the hierarchical classification of garbage, we introduce the idea of curriculum learning [12] whose basic idea is to organize objects in an order from simple to complex and to learn objects from easy to difficult, which helps understand more complex concepts through simple tasks learning.”
BackgroundLiu, X., Jin, K., Yang, Z., Yan, Y., Wang, S., Wang, Y., & Ye, J. (2022) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ac749b · Cited by 9
“In 2009, Bengio et al proposed the concept of curriculum learning, whosemain conceptual approach is to imitate the characteristics of human learning and learn the curriculum from simple to difficult (Bengio et al 2009).”
BackgroundLearning to Utilize Curiosity: A New Approach of Automatic Curriculum Learning for Deep RL
Lin, Z., Lai, J., Chen, X., Cao, L., & Wang, J. (2022) · Mathematics · doi:10.3390/math10142523 · Cited by 7
An Implicit Identity-Extended Data Augmentation for Low-Resolution Face Representation Learning
Low, C.-Y., & Teoh, A. B. J. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3201374 · Cited by 9
Filtering out Outliers in Learning to Rank
Marcuzzi, F., Lucchese, C., & Orlando, S. (2022) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3539813.3545127 · Cited by 7
“The following question arises: could we resort to curriculum learning [1] to properly exploit those hard instances? According”
Video-Guided Curriculum Learning for Spoken Video Grounding
Xia, Y., Zhao, Z. L., Ye, S., et al. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547996 · Cited by 7
“[2] propose curriculum leaning that gradually increased the training data complexity.”
BackgroundOnline Coordinated NFV Resource Allocation via Novel Machine Learning Techniques
Li, Z., Wu, L., Xiangyun, Z., et al. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3205900 · Cited by 7
“Inspired by curriculum learning [16], we use simple SFCRs to train agents at the beginning of training, generate effective policies for simple scenarios and then gradually increase the complexity of SFCRs.”
MethodM entor GNN: Deriving Curriculum for Pre-Training GNNs
Zhou, D., Zheng, L., Fu, D., Han, J., & He, J. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557393 · Cited by 32
“In practice, we gradually augment the value of λ1 + λ2 to enforce MentorGNN learning from the “easy” graph signals to the “hard” graph signals by mimicking the cognitive process of humans and animals [2].”
MethodCCL: Class-Wise Curriculum Learning for Class Imbalance Problems
Escudero-Viñolo, M., & López-Cifuentes, A. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897273 · Cited by 12
“Humans and animals learn better when the learning examples are not randomly presented but organized in a meaningful order that illustrates gradually more concepts, and gradually more complex ones [14].”
Background
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.