Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 3,101–3,200 of 5,240
Curriculum Contrastive Learning for Aspect-based Sentiment Analysis
Jian, Z., Wu, D. D., Zeng, X., Yao, J., Wang, M., & Wu, Q. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890648 · Cited by 4
“Bengio et al. [33] introduced CurL to enhance model performance by progressively training with easy-to-hard samples.”
Curriculum is more influential than haptic feedback when learning object manipulation
Ojaghi, P., Mir, R., Marjaninejad, A., Erwin, A., Wehner, M. J., & Valero‐Cuevas, F. J. (2025) · Science Advances · doi:10.1126/sciadv.adp8407 · Cited by 6
“From the traditional definitions of Vanilla or Progressive curriculum learning ( 67 , 68 ), one might assume that first learning to lift the ball (a form of grasp) is “easier” than rotating it, which involves a dynamic behavior ( 4 , 18 ) and a curriculum strategy in which rotation is learned only…”
The MacqD deep-learning-based model for automatic detection of socially housed laboratory macaques
Moat, G. J., Gaudet-Trafit, M., Paul, J., Bacardit, J., Hamed, S. B., & Poirier, C. (2025) · Scientific Reports · doi:10.1038/s41598-025-95180-x · Cited by 5
“59 , a strategy where models are trained by gradually increasing task complexity, mimicking human learning by starting with simpler concepts and progressing to more difficult ones.”
Focus on Hard Areas of Reconstruction: A Fine‐Grained Urban Flow Inference Framework
Zhang, Y., Yu, W., Zhao, X., He, Z., Zhang, W., & Lin, J. (2025) · Transactions in GIS · doi:10.1111/tgis.70039 · Cited by 3
“Specifically, two models including a flow reconstruction model and an encoder-dual decoder model are proposed, and two progressive stages in terms of curriculum strategy (Bengio et al. 2009) are designed to exploit explicitly spatial heterogeneity modeling.”
Structured experience shapes strategy learning and neural dynamics in the medial entorhinal cortex
Bowler, J. C., Azhar, D., Jensen, C. M., Lee, H., & Heys, J. G. (2026) · Nature Neuroscience · doi:10.1038/s41593-026-02409-7 · Cited by 5
“Notably, it has also been shown in silico that pretraining steps can bias dynamical systems to adopt similar strategies as those utilized by animals when solving complex problems 6,42,43 .”
From Easy to Hard: Building a Shortcut for Differentially Private Image Synthesis
Li, K., Gong, C., Li, X., Zhao, Y., Hou, X., & Wang, T. (2025) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp61157.2025.00217 · Cited by 15
“In Curriculum Learning [26], de-composing complex tasks into multiple steps and learning from easy to hard, are significantly useful in many machine learning tasks [27].”
Ahmed, F., Arsalan, M., Al-Ali, A. K., et al. (2025) · IEEE International Symposium on Computer-Based Medical Systems (CBMS) · doi:10.1109/cbms65348.2025.00175 · Cited by 3
“A curriculum learning strategy [39] is employed to gradually transition the training emphasis from segmentation specialization to full model collaboration to ensure robust performance in complex surgical scenarios.”
Xi, R., Fu, Z., Huang, N., Zhao, X., Zhang, Q., & Han, J. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3598924 · Cited by 5
“To achieve this, inspired by the principles of human learning curricula [23], [24] that gradually acquire knowledge and skills from simple to complex concepts, we propose a cross-modality self-paced association learning framework depicted in Fig.”
CLIMD: A Curriculum Learning Framework for Imbalanced Multimodal Diagnosis
Han, K., Lyu, C., Ma, L., Qian, C., Ma, S., Pang, Z., Chen, J., & Liu, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05182-0_7 · Cited by 8
A Multi-Modal Knowledge-Enhanced Framework for Vessel Trajectory Prediction
Yu, H., Li, T., Torp, K., & Jensen, C. S. (2025) · International Symposium on Spatial and Temporal Data (SSTD) · doi:10.1145/3748777.3748784 · Cited by 7
Integrating meta-information into recurrent neural network language models
Shi, Y., Larson, M., Pelemans, J., Jonker, C. M., Wambacq, P., Wiggers, P., & Demuynck, K. (2015) · Speech Communication · doi:10.1016/j.specom.2015.06.006 · Cited by 4
WITHDRAWN: I2VM: Incremental import vector machines
Roscher, R., Förstner, W., & Waske, B. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.08.008 · Cited by 3
Addanki, K., & Wu, D. (2014) · doi:10.3115/v1/w14-4013 · Cited by 3
“More recently, the probabilistic NNLMs of Bengio et al. (2003) and Bengio et al. (2009) follow in this vein.”
BackgroundSmart task orderings for active online multitask learning
Pang, P., An, J., Zhao, J., Li, X., Ban, T., Inoue, D., & Sarrafzadeh, H. (2014) · Unitec Research Bank (Unitec Institute of Technology) · Cited by 3
Robust ridge regression based on self-paced learning for multivariate calibration
Peng, J., & Tian, L. (2018) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2018.03.004 · Cited by 3
An end-to-end synthesis method for Korean text-to-speech systems
Choi, Y., Jung, Y., Kim, Y., Suh, Y., & Kim, H. (2018) · Phonetics and Speech Sciences · doi:10.13064/ksss.2018.10.1.039 · Cited by 2
How to Train Your Dragon: Best Practices in Pedestrian Classifier Training
Trichet, R., & Brémond, F. (2019) · IEEE Access · doi:10.1109/access.2019.2891950 · Cited by 3
“Finally, [6] proposed curriculum learning to progressively incorporate into the training set data”
BackgroundDeep Adversarial Reinforcement Learning for Object Disentangling
Laux, M., Arenz, O., Peters, J., & Pajarinen, J. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341578 · Cited by 5
“based tasks to the learning agent in an order of increasing complexity instead of presenting tasks randomly [24].”
Investigating Class-Level Difficulty Factors In Multi-Label Classification Problems
Marsden, M. A., McGuinness, K., Antony, J., Wei, H., Redzic, M., Tang, J., Hu, Z., Smeaton, A. F., & O'Connor, N. E. (2020) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme46284.2020.9102798 · Cited by 4
“Bengio et al. looked at difficulty in machine learning in the context of so called Curriculum Learning [6], in which a perceptron is trained on progressively more difficult samples over time, improving the overall convergence time and predictive performance of the model.”
BackgroundMethodInfluentialCMU-informedia @ TRECViD 2014 semantic indexing
Jiang, L., Chang, X., Mao, Z., Armagan, A., Lan, Z., Li, X., Yu, S.-I., Yang, Y., Meng, D., Şahin, P. D., et al. (2014) · Monash University Research Portal (Monash University) · Cited by 6
Estimation of Cardiac Valve Annuli Motion with Deep Learning
Kerfoot, E., King, C. E., Ismail, T., Nordsletten, D., & Miller, R. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_15 · Cited by 5
“rks dened for it. This step ensured that landmarks exhibiting lower accuracy would, at later stages in the training, become more common in the selected minibatches. This curriculum learning approach [1,2] forced the network to concentrate on those landmarks which were more dicult to accurately place compared to others…”
BackgroundDeep curriculum learning in task space for multi-class based mammography diagnosis
Luo, J., Arefan, D., Zuley, M. L., Sumkin, J. H., & Wu, S. (2022) · SPIE Medical Imaging: Computer-Aided Diagnosis · doi:10.1117/12.2612617 · Cited by 3
Learning to Augment Imbalanced Data for Re-ranking Models
Qiu, Z.-H., Jian, Y., Chen, Q.-G., et al. (2021) · Proceedings of the ... ACM International Conference on Information & Knowledge Management · doi:10.1145/3459637.3482364 · Cited by 3
“Such features are commonly used in curriculum learning [6, 29].”
BackgroundMulti-Task Long-Range Urban Driving Based on Hierarchical Planning and Reinforcement Learning
Zhang, X., Wu, M., Ma, H., Hu, T., & Yuan, J. (2021) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc48978.2021.9564705 · Cited by 3
Double Deep Q Learning with Gradient Biasing for Mobile Relay Beamforming Networks
Evmorfos, S., Diamantaras, K., & Petropulu, A. P. (2021) · 2018 52nd Asilomar Conference on Signals, Systems, and Computers · doi:10.1109/ieeeconf53345.2021.9723405 · Cited by 3
Composing Structure-Aware Batches for Pairwise Sentence Classification
Waldis, A., Beck, T., & Gurevych, I. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.239 · Cited by 3
“This idea is also at the core of Curriculum Learning (Bengio et al., 2009) where training instances are reordered according to their estimated difficulty.”
BackgroundMethodInfluentialCross-modal mask fusion and modality-balanced audio-visual speech recognition
Li, B., Wu, Z., & Wang, Y. (2022) · IEEE International Conference on Power, Intelligent Computing and Systems (ICPICS) · doi:10.1109/icpics55264.2022.9873574 · Cited by 3
“Existing methods treat hard examples and easy examples in the same way, which has difficulty in training[21] ( ( , , )) i i x y w Self-paced learning introduces a dichotomous variable i v , and the value of i v is either zero or one.”
MethodSelf-Supervised Fully Automatic Learning Machine for Intelligent Retail Container
Zhu, P., Sun, Y., Cao, B., Liu, X., Liu, X., & Hu, Q. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3204080 · Cited by 4
“Curriculum learning [26] proposes a training 152 strategy that first divides samples into different groups from 153 easy to difficult and then trains the model in turn.”
BackgroundJaensch, F., Kübler, K., Schwarz, E., & Verl, A. W. (2022) · Procedia CIRP · doi:10.1016/j.procir.2022.09.043 · Cited by 3
Voting-Based Contour-Aware Framework for Medical Image Segmentation
Deng, Q., Zhang, R., Li, S., Hong, J., Zhang, Y., Chu, C. W., & Shi, L. (2022) · Applied Sciences · doi:10.3390/app13010084 · Cited by 3
Dabane, G., Perrinet, L., & Daucé, E. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892313 · Cited by 7
“A curriculum learning training scheme [35] is used to train the networks, meaning that at the beginning of training, only small eccentricities with a fixed high contrast are used, then incrementally making the task harder throughout epochs.”
MethodTraining Techniques for Presence-Only Habitat Suitability Mapping with Deep Learning
Kellenberger, B., Cole, E., Marcos, D., & Tuia, D. (2022) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss46834.2022.9883627 · Cited by 6
“To counteract the long-tailed distribution, we employ curriculum learning [17], where the model is exposed to increasingly difficult (here: imbalanced) scenarios during training.”
MethodA Cooperation Graph Approach for Multiagent Sparse Reward Reinforcement Learning
Fu, Q., Qiu, T., Pu, Z., Yi, J., & Yuan, W. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9891991 · Cited by 4
“For some specific multiagent tasks, it is possible to use curriculum learning [13], [14] to train multiagents starting from a very small scale, then gradually expand the size of the team.”
BackgroundLifelong Fine-Grained Image Retrieval
Chen, W., Xu, H., Pu, N., Liu, Y., Lao, M., Wang, W., Liu, L., & Lew, M. S. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3222934 · Cited by 4
Parameter-efficient feature-based transfer for paraphrase identification
Liu, X., Rzepka, R., & Araki, K. (2022) · Natural Language Engineering · doi:10.1017/s135132492200050x · Cited by 3
“Curriculum learning (Bengio et al .”
MethodA deep reinforcement learning agent for geometry online tutoring
Xiao, Z., & Zhang, D. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01804-3 · Cited by 8
“Curriculum learning Curriculum learning [21] is inspired by the way humans acquire knowledge and has witnessed performance improvement in computation vision [22, 23] and natural language understanding [24].”
BackgroundSchema dependency-enhanced curriculum pre-training for table semantic parsing
Qin, B., Hui, B., Wang, L., Yang, M., Li, B., Huang, F., Si, L., Jiang, Q., & Li, Y. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110264 · Cited by 4
Emulating quantum dynamics with neural networks via knowledge distillation
Yao, Y., Cao, C., Haas, S., Agarwal, M., Khanna, D., & Abram, M. (2023) · Frontiers in Materials · doi:10.3389/fmats.2022.1060744 · Cited by 9
“To extract knowledge, we use the concept of curriculum learning Bengio et al. (2009). Namely, we construct a training set that allows a neural network to effectively learn the basic rules governing the physics of the quantum system. This procedure can also be viewed as knowledge distillation Gou et al. (2021) from a p…”
MethodInfluentialReference-Free Learning Bipedal Motor Skills via Assistive Force Curricula
Shi, F., Kojio, Y., Makabe, T., Anzai, T., Kojima, K., Okada, K., & Inaba, M. (2023) · Springer proceedings in advanced robotics · doi:10.1007/978-3-031-25555-7_21 · Cited by 9
Distribution Matching for Machine Teaching
Cao, X., & Tsang, I. W.-H. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3256412 · Cited by 4
“For instance, in curriculum learning [15], [30], a human can intelligently annotate or recognize examples when the teaching examples are not randomly presented but organized in a meaningful order.”
BackgroundModeling reciprocal adaptation in HCI: a Multi-Agent Reinforcement Learning Approach
Gupta, T., & Gori, J. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585913 · Cited by 3
“• The assistant selects a gain �� ∈ [1, 4] to modulate the user actions.”
MethodLiu, Z., Liu, B., Zhao, Z., Chu, Q., & Yu, N. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097020 · Cited by 4
“Curriculum learning [10] is a technique motivated by the cognitive process of humans that gradually learn from easy to complex samples.”
EuclidNets: An Alternative Operation for Efficient Inference of Deep Learning Models
Li, X., Parazeres, M., Oberman, A. M., Ghaffari, A., Asgharian, M., & Nia, V. P. (2023) · SN Computer Science · doi:10.1007/s42979-023-01921-y · Cited by 6
“Recently, homotopy was used in deep learning in the context of activation functions [37–40], loss functions [41], compression [42] and transfer learning [43].”
NC2T: Novel Curriculum Learning Approaches for Cross-Prompt Trait Scoring
Lee, Y., Jeong, S., Kim, H., et al. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3592027 · Cited by 3
“We believe that the key to addressing the problem of evaluating essays with limited data for various prompts in AES lies in designing a suitable curriculum learning (CL) [1].”
BackgroundZhang, H., Li, P., Liu, X., Yang, X. “., & An, L. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3606374 · Cited by 8
“One might also use curriculum learning [7] to select the most appropriate training samples at each training iteration.”
BackgroundT3S: Improving Multi-Task Reinforcement Learning with Task-Specific Feature Selector and Scheduler
Yu, Y., Yang, T., Lv, Y., Zheng, Y., & Hao, J. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191536 · Cited by 6
“Previous work also has shown that optimized task scheduling instead of uniform sampling can significantly improve model performance [27].”
Curriculum learning and evolutionary optimization into deep learning for text classification
Elías-Miranda, A. A., Vallejo-Aldana, D., Sánchez-Vega, F., López-Monroy, A. P., Rosales-Pérez, A., & Muñiz-Sánchez, V. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08632-8 · Cited by 4
“The main idea behind is to present the easy examples first to the computational model in order to obtain a better performance in the current task [11].”
BackgroundMethodInfluentialWeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning
Wu, W., Li, W., Xiao, X., Liu, J., Li, S., & Lyu, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.18 · Cited by 20
Data Cartography for Low-Resource Neural Machine Translation
Richburg, A., & Carpuat, M. J. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.410 · Cited by 4
“Some of the earlier works by Bengio et al. (2009); Cirik et al. (2016) show that ordering training samples by sentence length can decrease training speed or improve model performance.”
Kim, J., Koo, S., & Lim, H. (2023) · Electronics · doi:10.3390/electronics12194171 · Cited by 5
“For instance, in the student learning curricula, receiving only excessively challenging problems relative to students’ level can diminish the desire to achieve, whereas consistently encountering overly simple problems might lead to reduced interest [31,32].”
BackgroundDeng, J., Yao, H., & Shi, P. A. (2023) · Sensors · doi:10.3390/s23208406 · Cited by 3
REST: Drug-Drug Interaction Prediction via Reinforced Student-Teacher Curriculum Learning
Li, X., Qiu, Z., Zhao, X., Zhang, Y., Xing, C., & Wu, X. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615033 · Cited by 4
“Curriculum learning [1] aims at simulating the learning process from simple to difficult for human beings to increase the convergence speed and improve the generalization performance of machine learning models.”
BackgroundReinforcement Learning Based Path Tracking Control Method for Unmanned Bicycle on Complex Terrain
Huo, B., Yu, L., Liu, Y., & Sha, S. (2023) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon51785.2023.10312041 · Cited by 5
“To enable agents to complete path-tracking tasks, this paper adopts the idea of curriculum learning [19].”
MethodA robust optimization method for label noisy datasets based on adaptive threshold: Adaptive-k
Dedeoglu, E., Kesgin, H. T., & Amasyalı, M. F. (2023) · Frontiers of Computer Science · doi:10.1007/s11704-023-2430-4 · Cited by 4
“It is possible to associate this with curriculum learning, which reduces the priority of difficult samples and in some conditions, may improve vanilla performance [8, 9].”
BackgroundGuo, K., Diefenbach, D., Gourru, A., & Gravier, C. (2023) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai59109.2023.00032 · Cited by 12
“We also considered a curriculum learning approach [25], in which more simple QA pairs will be used and we would introduce more and more difficult QA samples as the training progresses.”
MethodA Model for Cognitively Valid Lifelong Learning
Say, H., & Öztop, E. (2023) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio58561.2023.10355028 · Cited by 5
“As it has been long known that curriculum learning facilitates faster neural network training [14], [15], recent work in deep reinforcement learning (RL) also often makes use of the idea [16].”
BackgroundFerKD: Surgical Label Adaptation for Efficient Distillation
Shen, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00160 · Cited by 5
“Some other techniques that are close to hard sample mining: (1) Curriculum learning [3, 48, 40]: it trains a model on easy examples first and then gradually increasing the difficulty of the examples over time.”
BackgroundScale-Invariant Reinforcement Learning in Real-Time Strategy Games
Lemos, M. L. H. D., Vieira, R. E. S., Tavares, A. R., Marcolino, L. S., & Chaimowicz, L. (2023) · Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) · doi:10.1145/3631085.3631337 · Cited by 4
Echoes of images: multi-loss network for image retrieval in vision transformers
Pundhir, A., Sagar, S., Singh, P., & Raman, B. (2024) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-024-03055-6 · Cited by 5
“To further augment the performance, we adopt the established two-branch network strategy to handle the joint task of image classification and retrieval [25].”
Behavior Analysis of Parent-Child Interactions from Text
Nikbakhtbideh, B., Duffett‐Leger, L., & Moshirpour, M. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00176 · Cited by 5
“Hence, the weighting method in our final submission made a balance during curriculum learning ( [19]), whereby the weighting strategy was only used in the first half of the training.”
MethodSequencing Initial Conditions in Physics-Informed Neural Networks
Hooshyar, S., & Elahi, A. (2024) · Journal of Chemistry and Environment · doi:10.56946/jce.v3i1.345 · Cited by 3
“The concept behind curriculum learning [50], inspired by Here, we again consider a one-dimensional convection problem with � = 15 and 푘 = 0.”
Data Aggregation (DAgger) Algorithm Using Adversarial Agent Policy for Dynamic Situations
Ahn, J., Shin, S., Koo, J., Kim, M., & Park, J. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-44981-9_8 · Cited by 3
Curriculum Learning-Based Multivariate Time Series Anomaly Detection
Dube, D. C., & Akar, M. (2024) · Conference of Open Innovations Association (FRUCT) · doi:10.23919/fruct61870.2024.10516396 · Cited by 4
“The concept of CL for machine learning was introduced in [11] and has since gained interest across several fields such as in federated face recognition [12] and natural language understanding [13] tasks.”
BackgroundMethodA Self-Training Approach Using Benchmark Dataset and Stereo-DSM for Building Extraction
Yuan, X., Tian, J., & Reinartz, P. H. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3412369 · Cited by 3
“Inspired by curriculum learning[38],analgorithmnamedCurriculumLabelingwasproposed toprogressivelyselecthardersamplesbyvaryingpercentile thresholds.”
DCLCSE: Dynamic Curriculum Learning Based Contrastive Learning of Sentence Embeddings
Liu, C., Zhang, D., & Wang, M. (2024) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2024.3423650 · Cited by 3
“Inspired by human learning process [13], [14], Ben-gio et al. [15] first introduced curriculum learning into machine learning, which is designed to arrange samples from easy to hard during the model learning process.”
Wang, S., Chen, J., Bai, B.-R., Fang, S., & Tsao, Y. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3426996 · Cited by 3
“This limitation, caused by variations in noise types, has been documented in previous studies [41], [42], [43].”
ResultOpen-vocabulary object detection via debiased curriculum self-training
Zhang, H., Guan, D., Ke, X., Saddik, A. E., & Lu, S. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124762 · Cited by 4
Joint Client-and-Sample Selection for Federated Learning via Bi-Level Optimization
Li, A., Wang, G., Hu, M., Sun, J., Zhang, L., Tuan, L. A., & Yu, H. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3455331 · Cited by 6
“Approaches based on curriculum learning (CL) [22], [23] attempt to train a model with easier samples in early training stages and gradually using more difficult training samples in later training stages.”
BackgroundGenerative AI for Energy: Multi-Horizon Power Consumption Forecasting using Large Language Models
Roitero, K., D'Abrosca, G., Zancola, A., Mea, V. D., & Mizzaro, S. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679933 · Cited by 5
Advancing Behavior Generation in Mobile Robotics through High-Fidelity Procedural Simulations
Kich, V. A., Bottega, J. A., Steinmetz, R., Grando, R. B., Yorozu, A., & Ohya, A. (2024) · IEEE RO-MAN · doi:10.1109/ro-man60168.2024.10731247 · Cited by 3
“This opens up possibilities for leveraging YamaS in various applications, including Deep-RL, LLM systems, and curriculum learning [24].”
Key-based data augmentation with curriculum learning for few-shot code search
Zhang, F., Peng, M., Wu, Q., et al. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10670-9 · Cited by 3
“Curriculum learning has demonstrated its ability to improve model generalization and convergence speed in computer vision and natural language processing [18].”
All-Weather Vehicle Detection and Classification with Adversarial and Semi-Supervised Learning
Huang, Y., & Lin, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78447-7_22 · Cited by 3
Enhancing few-shot learning using targeted mixup
Jnr., Y. D., & Kang, D.-K. (2025) · Applied intelligence (Boston) · doi:10.1007/s10489-024-06157-8 · Cited by 4
“Bengio et al. [3] published their work on CL in 2009.”
Tao, C., Kuang, D., Huang, Z., Peng, C., & Li, H. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3545760 · Cited by 8
“…have proven that this process optimization results in a better learning effect on HCSs; this is in line with the core idea of curriculum learning [17], [18], [19], which allows the model to first learn relatively simple samples so that it can form basic concepts and patterns and then gradually…”
Not all samples are equal: Boosting action segmentation via selective incremental learning
Huang, F., Chen, X.-D., Wu, W., & Ma, W. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110334 · Cited by 5
Curriculum Learning Algorithms for Reward Weighting in Sparse Reward Robotic Manipulation Tasks
Fele, B., & Babič, J. (2025) · IEEE Access · doi:10.1109/access.2025.3549639 · Cited by 5
“By initially focusing on simple and familiar tasks and gradually incorporating more complex and unfamiliar ones, CL can decrease the exploration space, prevent getting stuck in local optima, and expedite convergence [4].”
Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation
Wang, Q., Lv, Y., Mao, Y., Qu, Y. P., Xu, Y., & Ji, X. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3690624.3709337 · Cited by 18
What Comes After Transformers? A Selective Survey Connecting Ideas in Deep LearningGPT
Schneider, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-87330-0_4 · Cited by 17
Liu, Y., Kuang, Z., Zhang, H., Ma, Y., Ding, X., Huang, Y., Cai, C., & Li, X. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3578621 · Cited by 3
“This idea is first introduced in the work on curriculum learning (CL) by Bengio et al. [37] and is later proposed by Jeffery Dean et al. [38] in their work on deep learning in large-scale distributed systems.”
Knowledge-Aligned Domain Shift Tuning for Efficient Adaptation in Large Language Models
Kawamae, N. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737013 · Cited by 9
“Furthermore, the gradual decrease of 𝛼 is loosely inspired by curriculum learning [4], allowing the model to initially rely more on the pre-trained representation h for task adaptation, then shift towards the steered hidden state h as training proceeds.”
De Novo Molecular Design Enabled by Direct Preference Optimization and Curriculum Learning
Hou, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-06106-5_6 · Cited by 3
A quadruped robot designed to complete a marathon on a single battery charge
Lee, C., Youm, D., Park, J., Lee, J. H., Choi, S., Ji, G., Mun, J., Jung, M., Choi, H. C., Kim, H., et al. (2026) · Nature · doi:10.1038/s41586-026-11102-5 · Cited by 4
“…uniformly sampled within pre-defined ranges: v ω , , ~ (−1,4), (−1,1), (−1,1). y z (10) x As training progresses through curriculum learning 55 , the initial state distribution, velocity command ranges and terrain difficulty are gradually increased in both complexity and magnitude…”
Curriculum Design for Machine Learners in Sequential Decision Tasks
Peng, B., MacGlashan, J., Loftin, R., et al. (2017) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/3091125.3091403 · Cited by 3
Self-Paced Multi-View Clustering via a Novel Soft Weighted Regularizer
Huang, Z., Ren, Y., Liu, W., et al. (2019) · IEEE Access · doi:10.1109/access.2019.2954559 · Cited by 3
“To address the non-convexity issue, an effective and efficient way is to use curriculum learning [25] and self-paced learning [26].”
MethodDiscovering Biased News Articles Leveraging Multiple Human Annotations
Lazaridou, K., Löser, A., Mestre, M., & Naumann, F. (2020) · Language Resources and Evaluation · Cited by 4
Learning Arithmetic Operations With A Multistep Deep Learning
Nollet, B., Lefort, M., & Armetta, F. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206963 · Cited by 4
“For instance, this can be done with curriculum learning where tasks of increasing complexity are successively presented to the network [23].”
Difficulty Translation in Histopathology Images
Wei, J., Suriawinata, A. A., Liu, X., Ren, B., Nasir-Moin, M., Tomita, N., Wei, J., & Hassanpour, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59137-3_22 · Cited by 6
“Future research could explore difficulty translation in the context of curriculum learning [29] or defending against adversarial attacks [30].”
BackgroundVec2UAge: Enhancing underage age estimation performance through facial embeddings
Anda, F., Dixon, E. M., Bou‐Harb, E., Le‐Khac, N., & Scanlon, M. (2021) · Forensic Science International Digital Investigation · doi:10.1016/j.fsidi.2021.301119 · Cited by 7
“If the complexity of the problem is gradually increased, this is known as curriculum learning where the speed of convergence of the training process is increased (Bengio et al., 2009).”
BackgroundEvolutionary Game Theory Squared: Evolving Agents in Endogenously Evolving Zero-Sum Games
Skoulakis, S., Fiez, T., Sim, R., Piliouras, G., & Ratliff, L. J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17352 · Cited by 12
“For example, in adversarial learning and curriculum learning (Huang et al., 2011; Bengio et al., 2009), the difficulty of the game can increase over time by exactly focusing on the settings where the agent has performed the weakest.”
BackgroundFacial Age Estimation by Curriculum Learning
Wang, W., Ishikawa, T., & Watanabe, H. (2020) · IEEE Global Conference on Consumer Electronics (GCCE) · doi:10.1109/gcce50665.2020.9291929 · Cited by 3
“The process of curriculum learning, in brief, is distributing the training data into several complex levels, allowing the model to learn simpler data at first, which can avoid introducing noisy early[6].”
BackgroundMethodExtracting Search Tasks from Query Logs Using a Recurrent Deep Clustering Architecture
Lugo, L., Moreno, J. G., & Hubert, G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72113-8_26 · Cited by 5
Stratified Multi-Task Learning for Robust Spotting of Scene Texts
Dasgupta, K., Das, S. K., & Bhattacharya, U. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9411951 · Cited by 4
“Apart from that, we employ curriculum learning [47] concerning the complexity of images to enable the feature learning invariant in terms of degradation, different lighting conditions, and image complexities.”
MethodPA-FlowNet: Pose-Auxiliary Optical Flow Network for Spacecraft Relative Pose Estimation
Chen, Z., Chen, P.-H., Kuan-Wen, C., & Chan, C.-Y. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413201 · Cited by 7
“Moreover, we introduced the foreground-attention approach inspired by Derakhshani et al. [6] with the concept of curriculum learning [2].”
BackgroundMethodApplying End-to-End Trainable Approach on Stroke Extraction in Handwritten Math Expressions Images
Moussa, E. M., Lelore, T., & Mouchère, H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86334-0_29 · Cited by 3
Exploring Contextual-Aware Representation and Linguistic-Diverse Expression for Visual Dialog
Li, X., Gao, L., Zhao, L., & Song, J. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475360 · Cited by 3
“[2], which requires the model to learn from simple problems and then gradually diverge to difficult problems.”
BackgroundLin, J., Ye, C., Liu, W., Ren, S., Wang, Y., Ma, W., Xu, B., & Ding, Y. (2022) · Wireless Communications and Mobile Computing · doi:10.1155/2022/3017149 · Cited by 4
“The system which is called FaceNet utilizes triplets of face patches based on the method in [30] and achieves state-of-the-art face detection performance on the LFW dataset.”
MethodRetractedFinding Age Path of Self-Paced Learning
Gu, B., Zhai, Z., Li, X., & Huang, H. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00025 · Cited by 4
CLC: Noisy Label Correction via Curriculum Learning
Lee, J., Lim, H., & Chung, K. (2021) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci50451.2021.9660078 · Cited by 3
Crowdsourcing with Self-paced Workers
Kang, X., Yu, G., Domeniconi, C., Wang, J., Guo, W., Ren, Y., & Cui, L. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00038 · Cited by 4
Towards Better Generalization of Deep Neural Networks via Non-Typicality Sampling Scheme
Peng, X., Wang, F., & Li, L. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3147031 · Cited by 3
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.