Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 1,501–1,600 of 5,240
Incorporating Humanoid Artificial Intelligence (AI) Robots into Early Childhood Education
Lee, J., Jo, J., Lee, J. O., & Kim, S. H. (2024) · Early Childhood Education Journal · doi:10.1007/s10643-024-01690-0 · Cited by 28
“This involvement, as highlighted by Bengio et al. (2009), is key to ensuring that AI systems exhibit behaviors more similar to those of humans.”
A comprehensive survey of crowd density estimation and counting
Wang, M., Zhou, X., & Chen, Y. (2025) · IET Image Processing · doi:10.1049/ipr2.13328 · Cited by 32
Zhou, R., Huang, J., Li, M., Li, H., Cao, H., & Song, X. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103188 · Cited by 24
Integrating language into medical visual recognition and reasoning: A survey
Lu, Y., & Wang, A. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103514 · Cited by 48
Advances in Continual Graph Learning for Anti‐Money Laundering Systems: A Comprehensive Review
Deprez, B., Wei, W., Verbeke, W., Baesens, B., Mets, K., & Verdonck, T. (2025) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.70040 · Cited by 32
“Curriculum Learning , coined by Bengio et al. [6], determines that knowledge can be optimally acquired if tasks are learned in ascending order of difficulty.”
Plenty is Plague: Fine-Grained Learning for Visual Question Answering
Zhou, Y., Ji, R., Sun, X., Su, J., Meng, D., Gao, Y., & Shen, C. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2956699 · Cited by 19
Ursuleanu, T. F., Luca, A. R., Gheorghe, L., Grigorovici, R., Iancu, S., Hlusneac, M., Preda, C., & Grigorovici, A. (2021) · Diagnostics · doi:10.3390/diagnostics11081373 · Cited by 26
Unified curiosity-Driven learning with smoothed intrinsic reward estimation
Huang, F., Li, W., Cui, J., Fu, Y., & Li, X. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108352 · Cited by 18
Build Your Own Bundle - A Neural Combinatorial Optimization Method
Deng, Q., Wang, K., Zhao, M., Wu, R., Ding, Y., Zou, Z., Shang, Y., Tao, J., & Fan, C. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475440 · Cited by 28
“Under the setting of reinforcement learning, curriculum learning [6] is a methodology to optimize the order in which experience is accumulated by the agent, to increase performance or training speed on a set of final tasks.”
MethodFrom Less to More: Progressive Generalized Zero-Shot Detection With Curriculum Learning
Liu, J., Chen, Y., Liu, H., Zhang, H., & Zhang, Y. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3151073 · Cited by 20
“It is first proposed in [26] for imitating people’s ability to learn gradually as they grow up.”
BackgroundA semi-supervised learning method for surface defect classification of magnetic tiles
Liu, T., & Ye, W. (2022) · Machine Vision and Applications · doi:10.1007/s00138-022-01286-x · Cited by 18
“Curriculum learning [25] is a training strategy in machine learning that allows the model to learn easy samples at first and then gradually learn hard ones.”
BackgroundCross-Modal Retrieval for Motion and Text via DropTriple Loss
Yan, S., Liu, Y., Wang, H., Du, X., Liu, M., & Liu, H. (2023) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3595916.3626459 · Cited by 30
“To address this, we adopt a curriculum learning [1] strategy mentioned in [4]: before using these two losses, we warm-up the entire model for 𝜌 epochs using the SH Loss (Eq.”
Hu, Z., Gao, X., Wan, K., Wang, Q., & Zhai, Y. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3285595 · Cited by 33
“Many studies have made efforts to solve these problems: Ren et al. [30] proposed a deep curriculum reinforcement learning (DCRL) method that combines DRL with curriculum learning (CL) [31] to adaptively select appropriate experiences based on the complexity of each experience.”
MethodNew Insights on Relieving Task-Recency Bias for Online Class Incremental Learning
Liang, G., Chen, Z., Chen, Z., Ji, S., & Zhang, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3325651 · Cited by 23
“Then, based on curriculum learning [19], we design a revised focal loss which always pays more attention to samples located at current ASI.”
BackgroundMethodAutomated Deep Learning: Neural Architecture Search Is Not the End
Dong, X., Kedziora, D. J., Musiał, K., & Gabryś, B. (2024) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000119 · Cited by 53
“Batch size is a classic hyperparameter to consider, but even the order of data ingestion matters [22].”
BackgroundKesgin, H. T., & Amasyalı, M. F. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100071 · Cited by 24
From MNIST to ImageNet and back: benchmarking continual curriculum learning
Faber, K., Żurek, D., Pietroń, M., Japkowicz, N., Vergari, A., & Corizzo, R. (2024) · Machine Learning · doi:10.1007/s10994-024-06524-z · Cited by 26
“…curriculum learning—learning on tasks of increasing complexity—in conventional machine learning research showcased that significant improvements in generalization can be achieved, increasing the speed of convergence of the training process (Bengio et al., 2009; Gao et al., 2022; Song et al., 2020).”
PathRL: An End-to-End Path Generation Method for Collision Avoidance via Deep Reinforcement Learning
Yu, W., Peng, J., Qiu, Q., Wang, H., Zhang, L., & Ji, J. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610107 · Cited by 20
“Therefore, we employ curriculum learning [11] to assist and speed up the training process.”
MethodLi, D., Tang, J., Zhou, B., Cao, P., Hu, J. S., Leung, M.-F., & Wang, Y. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3456843 · Cited by 26
“The concept of Curriculum Learning (CL) was first introduced by Bengio et al. [31].”
Friston, K. J., Costa, L. D., Tschantz, A., Kiefer, A. B., Salvatori, T., Neacsu, V., Koudahl, M., Heins, C., Sajid, N., Marković, D., et al. (2024) · Biological Psychology · doi:10.1016/j.biopsycho.2024.108891 · Cited by 46
An Adaptive Resonance Theory account of the implicit learning of orthographic word forms
Glotin, H., Warnier, P., Dandurand, F., Dufau, S., Lété, B., Touzet, C. F., Ziegler, J. C., & Grainger, J. (2009) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2009.11.003 · Cited by 20
“Finally, more attention will be dedicated to curriculum learning (Bengio et al., 2009) in the near future.”
BackgroundCross-Paced Representation Learning With Partial Curricula for Sketch-Based Image Retrieval
Xu, D., Alameda-Pineda, X., Song, J., Ricci, E., & Sebe, N. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2837381 · Cited by 25
“(Email: xavier.alameda-pineda@inria.fr) Recent research efforts to overcome the problems associated to local optima resulted in two orthogonal trends: self-paced learning (SPL) [14] and curriculum learning (CL) [15].”
BackgroundMethodDecoding Strategies for Neural Referring Expression Generation
Zarrieß, S., & Schlangen, D. (2018) · International Conference on Natural Language Generation · doi:10.18653/v1/w18-6563 · Cited by 24
“A possible explanation is that our REG models benefit from a two-stage optimization procedure, similar to curriculum learning (Bengio et al., 2009).”
MethodIntegrating a Path Planner and an Adaptive Motion Controller for Navigation in Dynamic Environments
Zeng, J., Qin, L., Hu, Y., Yin, Q., & Hu, C. (2019) · Applied Sciences · doi:10.3390/app9071384 · Cited by 19
MSURU: Large Scale E-commerce Image Classification with Weakly Supervised Search Data
Tang, Y., Borisyuk, F., Malreddy, S., Li, Y., Liu, Y., & Kirshner, S. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330696 · Cited by 22
Jiang, N., Jin, S., & Zhang, C. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.024 · Cited by 21
“Curriculum learning [29] makes the agent gradually learn to master a challenging task by constructing curricula of easy-to-hard or clear-to-ambiguous tasks.”
BackgroundMethodImproving Anaphora Resolution in Neural Machine Translation Using Curriculum Learning.
Stojanovski, D., & Fraser, A. (2019) · Cited by 29
A Sentiwordnet Strategy for Curriculum Learning in Sentiment Analysis
Rao, V. A., Anuranjana, K., & Mamidi, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-51310-8_16 · Cited by 23
“Curriculum Learning (CL) as defined by [3] introduce and formulate this concept from Cognitive Science to a machine learning setting.”
BackgroundSaha, A., Prasad, P. W. C., & Thabit, A. (2020) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi45749.2020.9098344 · Cited by 29
“By initializing an equivalent encoder for lesion classification using these new weights, we can relay the semantic structural information required to segment skin lesions from normal skin, as a foundation for the more complex features required to be learned in order to discriminate between different types of skin lesi…”
MethodBagi, R., Dutta, T., & Gupta, H. P. (2020) · IEEE Access · doi:10.1109/access.2020.3002808 · Cited by 21
“The end-to-end training use curriculum learning [88] technique to train the model from gradual to complex data efficiently.”
BackgroundAgent Coordination in Air Combat Simulation using Multi-Agent Deep Reinforcement Learning
Källström, J., & Heintz, F. (2020) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc42975.2020.9283492 · Cited by 21
“One technique for addressing this challenge is curriculum learning [23].”
BackgroundC_CART: An instance confidence-based decision tree algorithm for classification
Yu, S., Li, X., Wang, H., Zhang, X., & Chen, S. (2021) · Intelligent Data Analysis · doi:10.3233/ida-205361 · Cited by 21
“Moreover, Bengio et al. [14] also emphasized that those “hard” instances were probably useless outliers or noisy instances that confused the classifiers so that the right location of the decision surface could not be established.”
Self-Guided Curriculum Learning for Neural Machine Translation
Zhou, L., Ding, L., Duh, K., Watanabe, S., Sasano, R., & Takeda, K. (2021) · International Conference on Spoken Language Translation (IWSLT) · doi:10.18653/v1/2021.iwslt-1.25 · Cited by 18
A novel approach for left ventricle segmentation in tagged MRI
Zou, X., Wang, Q., & Luo, T. (2021) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2021.107416 · Cited by 20
Integrating Iterative Machine Teaching and Active Learning into the Machine Learning Loop
Mosqueira-Rey, E., Alonso-Ríos, D., & Baamonde-Lozano, A. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.08.057 · Cited by 26
“…is focused on the e ff ectiveness of the learning process, placing special emphasis on how the teacher filters the training set and gives the student the most relevant examples depending on the moment of learning in which the student is, something that is closely related to curriculum learning [6].”
BackgroundMethodPruning-Guided Curriculum Learning for Semi-Supervised Semantic Segmentation
Kong, H., Lee, G., Kim, S., & Lee, S. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00586 · Cited by 25
“Curriculum learning [3] is a training strategy that gradually incorporates easier to harder samples during training, thereby imitating the meaningful learning order in human curricula.”
BackgroundDEJONG, G. F. (2014) · Chapman & Hall/CRC eBooks · doi:10.1201/9780429171529-37 · Cited by 17
Curriculum Learning for Handwritten Text Line Recognition
Louradour, J., & Kermorvant, C. (2014) · IAPR International Workshop on Document Analysis Systems (DAS) · doi:10.1109/das.2014.38 · Cited by 23
“This idea has been exploited in classification [9], grammar induction [10], [11], robotics [12], cognitive science [13] and human teaching [14].”
BackgroundMethodInfluentialSelf-citationHyperspectral Image Classification with Tensor-Based Rank-R Learning Models
Makantasis, K., Voulodimos, A. S., Doulamis, A. D., et al. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803268 · Cited by 18
“Adaptive Moment Estimation (Adam) [18] with learning rate equal to 0.”
BackgroundReconstruction for Feature Disentanglement in Pose-invariant Face RecognitionPreprint
Peng, X., Yu, X., Sohn, K., et al. (2017) · arXiv (Cornell University) · Cited by 18
ViZDoom Competitions: Playing Doom from PixelsPreprint
Wydmuch, M., Kempka, M., & Jaśkowski, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03470 · Cited by 18
“Tian)—the winning bot of Track 1 was trained with a variant of the A3C algorithm [19] with curriculum learning [2].”
MethodDynamic Computational Time for Visual AttentionPreprint
Li, Z., Yang, Y. P., Liu, X., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10332 · Cited by 18
“We address this problem with curriculum learning [13].”
BackgroundSPLSN: An efficient tool for survival analysis and biomarker selection
Huang, H., Peng, X., & Liang, Y. Q. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22532 · Cited by 17
“SPL, 43 also known as curriculum learning stimulated by how human learning is.”
BackgroundA Probabilistic Formulation for Meta-Weight-Net
Zhao, Q., Shu, J., Yuan, X., Liu, Z., & Meng, D. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3105104 · Cited by 23
“In the future, we will try to introduce more useful knowledge to the probabilistic weight function, such as curriculum learning [74], by designing proper priors.”
MethodBagi, R., Dutta, T., Nigam, N., Verma, D., & Gupta, H. P. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3117793 · Cited by 18
Immersive audio-visual scene reproduction using semantic scene reconstruction from 360 cameras
Kim, H., Remaggi, L., Dourado, A., de Campos, T. E., Jackson, P. J. B., & Hilton, A. (2021) · Virtual Reality · doi:10.1007/s10055-021-00594-3 · Cited by 22
“The edge volume is generated from the edge map as a voxel The one cycle learning policy (Smith 2018) combined with curriculum learning (Bengio et al. 2009) and simulated annealing (Aarts and Korst 1989) is used for the training stage.”
MethodPre-training a BERT with Curriculum Learning by Increasing Block-Size of Input Text
Nagatsuka, K., Broni-Bediako, C., & Atsumi, M. (2021) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-072-4_112 · Cited by 19
“Most studies in the field of CL for NLP have proposed variety of difficulty measure by leveraging heuristics of the target tasks with neural networks (Bengio et al., 2009; Kocmi and Bojar, 2017; So-viany et al., 2021; Spitkovsky et al., 2009; Cirik et al., 2016; Rajeswar et al., 2017).”
BackgroundMethodInfluentialAn Effective Conversion of Visemes to Words for High-Performance Automatic Lipreading
Fenghour, S., Chen, D., Guo, K., Li, B., & Xiao, P. (2021) · Sensors · doi:10.3390/s21237890 · Cited by 17
“Moreover, due to the curriculum learning strategy deployed for training the GRU network, it is better at recognising shorter N-grams [52–54].”
BackgroundA simple denoising approach to exploit multi-fidelity data for machine learning materials properties
Liu, X., Breuck, P.-P. D., Wang, L., & Rignanese, G. (2022) · npj Computational Materials · doi:10.1038/s41524-022-00925-1 · Cited by 21
“31 Some other works rely on curriculum learning 32 to gradually train the model using the complete dataset ordered in a meaningful sequence.”
Jinsi, O., Henderson, M. M., & Tarr, M. J. (2023) · PLoS ONE · doi:10.1371/journal.pone.0280145 · Cited by 24
“[49], more effective learning can be facilitated when inputs are ordered with a gradual increase in both number and complexity.”
BackgroundDing, Y., Liang, J., Jiang, B., Zheng, A., & He, R. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3294963 · Cited by 20
“Curriculum learning (CL) [60] is a training strategy that trains a model from easy to hard, which simulates the learning principle of humans and animals.”
BackgroundGong, H., Zhang, Y., Dong, C., Wang, Y., Chen, G., Liang, B., Li, H., Liu, L., Xu, J., & Li, G. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad589 · Cited by 17
“Curriculum learning (CL) (Bengio et al. 2009) stems from the idea that learning from easy to hard could improve the generalization ability of the model.”
BackgroundA Survey of Dataset Refinement for Problems in Computer Vision Datasets
Wan, Z., Wang, Z., Chung, C., & Wang, Z. (2024) · ACM Computing Surveys · doi:10.1145/3627157 · Cited by 32
“Inspired by the meaningful learning order in the human curriculum, curriculum learning (CL) [11], a classical data sampling strategy, proposes a meaningful sequence of data to train a model, i.”
BackgroundCDFSL-V: Cross-Domain Few-Shot Learning for Videos
Samarasinghe, S., Rizve, M. N., Kardan, N., & Shah, M. A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01069 · Cited by 28
“Curriculum Learning Curriculum learning involves prioritizing easier samples (or tasks) during training before increasing the weights of the more difficult samples [2].”
BackgroundLiao, K., Nie, L., Lin, C., Zheng, Z., & Zhao, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00991 · Cited by 30
“The general paradigm of curriculum learning [3] introduces a heuristic and purposeful strategy for training a learning model.”
BackgroundUnified ABSA via Annotation-Decoupled Multi-Task Instruction Tuning
Wang, Z., Xia, R., & Yu, J. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3392836 · Cited by 45
“Some other approaches [62], [63] can be considered to improve multi-task learning performance, and we leave these explorations to future work.”
BackgroundLarge Language Model-based Test Case Generation for GP Agents
Jorgensen, S., Nadizar, G., Pietropolli, G., Manzoni, L., Medvet, E., O’Reilly, U.-M., & Hemberg, E. (2024) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3638529.3654056 · Cited by 21
“CL, introduced by Elman [16] but formally defined by Bengio et al. [5], is a training strategy that mimics the progressive learning pattern observed in animals and humans [71].”
Curriculum Point Prompting for Weakly-Supervised Referring Image Segmentation
Dai, Q., & Yang, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01301 · Cited by 41
“Curriculum Learning for RIS: Curriculum learning proposed by [1], similar to the natural learning process, gradually increases the complexity of training data during the training to enhance the model’s capability.”
BackgroundObject and Action Classification with Latent Variables
Bilen, H., Namboodiri, V. P., & Gool, L. V. (2011) · British Machine Vision Conference (BMVC) · doi:10.5244/c.25.17 · Cited by 22
“Recent work [1] proposes an iterative approach to find better local minima within shorter convergence times for non-convex optimisation problems.”
BackgroundOrdinal Multi-Task Part Segmentation With Recurrent Prior Generation
Zhao, Y., Li, J., Zhang, Y., Song, Y., & Tian, Y. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2953854 · Cited by 18
“Inspired by Curriculum Learning [57], [58], learning tasks in a proper order ben-efits the learning procedure, yielding in much better results.”
MethodVehicle re-identification in tunnel scenes via synergistically cascade forests
Zhu, R., Fang, J., Li, S., Wang, Q., Xu, H., Xue, J., & Yu, H. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.11.069 · Cited by 17
“Therefore, in this work, we also want to check whether the performance can be boosted by some easy-to-difficult learning methods, such as curriculum learning [52,53] .”
BackgroundGaussian Pyramid of Conditional Generative Adversarial Network for Real-World Noisy Image Denoising
Ma, R., Zhang, B., & Hu, H. (2020) · Neural Processing Letters · doi:10.1007/s11063-020-10215-w · Cited by 20
One-step spectral clustering based on self-paced learning
Tong, T., Gan, J., Wen, G., et al. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.03.035 · Cited by 17
Webly Supervised Knowledge Embedding Model for Visual Reasoning
Zheng, W., Yan, L., Gou, C., & Wang, F. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01246 · Cited by 21
“Thus, during the second stage, we employ the strategy of curriculum learning [3] for training.”
MethodSensor Path Planning Using Reinforcement Learning
Hoffmann, F., Charlish, A., Ritchie, M., & Griffiths, H. (2020) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion45008.2020.9190242 · Cited by 22
“Curriculum learning is based on the idea of letting a machine learning algorithm first learn simple tasks, before moving to more complex tasks [18], [19].”
BackgroundDeep Reinforcement Learning for Navigation in Cluttered Environments
Regier, P., Gesing, L., & Bennewitz, M. (2020) · doi:10.5121/csit.2020.101117 · Cited by 17
App-Aware Response Synthesis for User Reviews
Farooq, U., Siddique, A. B., Jamour, F., Zhao, Z., & Hristidis, V. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9377983 · Cited by 22
“Whereas curriculum learning [14] algorithm passes the current prediction of the decoder to the next timestep to minimize the incompatibility of the train and test setups, and enables the model to correct itself, but the model may take longer to train and converge.”
BackgroundOptimizing Data Usage for Low-Resource Speech Recognition
Qian, Y., & Zhou, Z. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3140552 · Cited by 24
Adaptive Curriculum Learning for Video Captioning
Li, S., Yang, B., & Zou, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3160451 · Cited by 20
“Research on curriculum learning , a training strategy that is inspired by the human learning process and can be traced back to [14], believes that deep neural models can achieve better performance if the training examples are presented in a meaningful order, i.e., from easy ones to gradually…”
BackgroundSmith, M. R., & Martinez, T. R. (2014) · Computational Intelligence · doi:10.1111/coin.12047 · Cited by 19
“Curriculum learning (CL) was recently formalized by Bengio et al. (2009) as a means of using an ordering of the training data from simplest to most complex to train a learning algorithm.”
MethodBalanced self-paced learning with feature corruption
Ren, Y., Zhao, P., Xu, Z., & Yao, D. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966104 · Cited by 17
“[1] to handle difficult learning problems.”
ResultAn Optimization Framework for Task Sequencing in Curriculum Learning
Foglino, F., Christakou, C. C., & Leonetti, M. (2019) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2019.8850690 · Cited by 25
“Curriculum Learning has initially been introduced in supervised learning [13].”
BackgroundThe Challenge of Modeling the Acquisition of Mathematical Concepts
Testolin, A. (2020) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2020.00100 · Cited by 19
“Taking inspiration from the notions of transfer learning and curriculum learning used in machine learning (Bengio et al., 2009) and from shaping procedures used in animal conditioning (Skinner, 1953), these issues can be mitigated by decomposing the task into simpler sub-tasks.”
MethodA Formal Framework to Characterize Interpretability of ProceduresPreprint
Dhurandhar, A., Iyengar, V. S., Luss, R., & Shanmugam, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02952 · Cited by 57
“Weighting examples to improve models has been used before, although their general setup and motivation is different, for instance curriculum learning (CL) [5] and boosting [21].”
Soviany, P., & Ionescu, R. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.08707 · Cited by 32
“Based on the principles of curriculum learning [5], we hypothesize that using more complex (two-stage) object detectors for difficult images and less complex (single-stage) detectors for easy images will provide an optimal trade-off between accuracy and speed, without ever having to change anything about the object de…”
BackgroundMethodLearning a Deep ConvNet for Multi-label Classification with Partial LabelsPreprint
Durand, T., Mehrasa, N., & Mori, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.09720 · Cited by 17
“The idea of Curriculum Learning is inspired by the way humans learn: start to learn with easy samples/subtasks, and then gradually increase the difficulty level of the samples/subtasks.”
BackgroundMethodInfluentialDistribution Density, Tails, and Outliers in Machine Learning: Metrics and ApplicationsPreprint
Carlini, N., Erlingsson, Ú., & Papernot, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.13427 · Cited by 74
“While Bengio et al. (2009) assumed the existence of hard-coded curriculum labels in the dataset, Chin & Liang (2017) sample an order for the training set by assigning each point a sampling probability proportional to its leverage score—the distance between the point and a linear model fitted to the…”
BackgroundLearning Not to Learn in the Presence of Noisy LabelsPreprint
Ziyin, L., Chen, B., Wang, R., Liang, P. P., Salakhutdinov, R., Morency, L., & Ueda, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06541 · Cited by 25
“More importantly, since different data points are learned at potentially different speeds (Bengio et al., 2009), we propose a rule to automatically set λ i adaptively as a function of each data point (rather than a general λ ).”
Solving Differential Equations Using Neural Network Solution BundlesPreprint
Flamant, C., Protopapas, P., & Sondak, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.14372 · Cited by 19
“This pitfall can be avoided by applying curriculum learning[19] to the training process.”
BackgroundThe Next Big Thing(s) in Unsupervised Machine Learning: Five Lessons from Infant LearningPreprint
Zaadnoordijk, L., Besold, T. R., & Cusack, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.08497 · Cited by 17
“The general idea of applying a structured learning curriculum—as naturally experienced by infants—to ML systems had already been put forward by [43] and in the Deep Learning literature was prominently addressed under the headline of curriculum learning [12].”
BackgroundBeBold: Exploration Beyond the Boundary of Explored RegionsPreprint
Zhang, T., Xu, H., Wang, X., Wu, Y., Keutzer, K., Gonzalez, J. E., & Tian, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.08621 · Cited by 49
“Curriculum learning (Bengio et al., 2009) has also been used to solve hard exploration environments.”
BackgroundThe Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT ModelsPreprint
Li, C., Zhang, M., & He, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06084 · Cited by 17
“Curriculum Learning: Our method can be viewed as a kind of curriculum learning (CL) [12, 39, 2], which presents easier/simpler examples earlier during training and gradually increases the sample difficulties 2 .”
Curriculum-Based Federated Learning for Machine Fault Diagnosis With Noisy Labels
Sun, W., Yan, R., Jin, R., Zhao, R., & Chen, Z. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3435449 · Cited by 20
Learning to Cooperate in Decentralized Multi-robot Exploration of Dynamic Environments
Geng, M., Zhou, X., Ding, B., Wang, H., & Zhang, L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-04239-4_4 · Cited by 16
A New Multi-choice Reading Comprehension Dataset for Curriculum Learning.
Liang, Y., Li, J., & Yin, J. P. (2019) · Asian Conference on Machine Learning · Cited by 52
Sequential Domain Adaptation through Elastic Weight Consolidation for Sentiment Analysis
Madasu, A., & Vijjini, A. R. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412617 · Cited by 25
“Curriculum Learning[5] proposes that training a model such that it is provided easier samples first leads to better generalization.”
BackgroundTime-Consistent Self-Supervision for Semi-Supervised Learning
Zhou, T., Wang, S., & Bilmes, J. (2020) · International Conference on Machine Learning · Cited by 64
Dynamic Data Selection for Curriculum Learning via Ability Estimation
Lalor, J. P., & Yu, H. Q. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.48 · Cited by 17
“For example, sentence length is often used as a proxy for difficulty in NLP tasks (Bengio et al., 2009; Platanios et al., 2019).”
BackgroundMethodResultInfluentialSiame-se(3): regression in se(3) for end-to-end visual servoing
Felton, S., Fromont, É., & Marchand, É. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561488 · Cited by 17
“To remedy this, we use curriculum learning [2]: we progressively add ramp up augmentation over the first 10 epochs.”
MethodReinforced Curriculum Learning For Autonomous Driving In Carla
Anzalone, L., Barra, S., & Nappi, M. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506673 · Cited by 32
Nearest-Neighbor-based Collision Avoidance for Quadrotors via Reinforcement Learning
Ourari, R., Cui, K., Elshamanhory, A., & Koeppl, H. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812221 · Cited by 24
“learning [38] to improve learning performance, increasing the number of agents continuously while training.”
BackgroundCrossroads, Buildings and Neighborhoods: A Dataset for Fine-grained Location Recognition
Chen, P., Xu, H., Zhang, C., & Huang, R. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.243 · Cited by 51
“We follow the curriculum designing approach introduced by Bengio et al. (2009”
BackgroundMethodInfluentialWeakly-supervised Temporal Path Representation Learning with Contrastive Curriculum Learning
Yang, B., Guo, C., Hu, J., Yang, B., Tang, J., & Jensen, C. S. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00260 · Cited by 45
Feature-Balanced Loss for Long-Tailed Visual Recognition
Li, M., Cheung, Y., & Jiang, J. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9860003 · Cited by 20
“riculum learning [7] strategy to gradually increase the classbased stimulus so that the network initially concentrates on the head classes, and then gradually shifts its attention to the tail classes as the training progresses.”
BackgroundMethodDivergence-Driven Consistency Training for Semi-Supervised Facial Age Estimation
Bao, Z., Tan, Z., Wan, J., Ma, X., Guo, G., & Lei, Z. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3218431 · Cited by 22
Pedestrian Simulation with Reinforcement Learning: A Curriculum-Based Approach
Vizzari, G., & Cecconello, T. (2022) · Future Internet · doi:10.3390/fi15010012 · Cited by 20
“…most important aspect of this study, however, is the adopted learning process: we defined a particular training curriculum (a concept introduced by [8]), a set of scenarios growing in difficulty supporting the incremental acquisition of proper orientation, walking, and pedestrian interaction…”
BackgroundDeep Metric Learning with Adaptively Composite Dynamic Constraints
Zheng, W., Lu, J., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3234536 · Cited by 16
“Inspired by curriculum learning [2], Zheng et al.”
MethodDirection and Residual Awareness Curriculum Learning Network for Rain Streaks Removal
Chang, Y., Chen, M., Yu, C., Li, Y., Chen, L., & Yan, L. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3227730 · Cited by 18
“The curriculum learning [41], [42], [43] is to learn the samples that are not randomly presented but organized in a meaningful order, in which a model is learned gradually from easy to complex samples.”
BackgroundFocused Adaptation of Dynamics Models for Deformable Object Manipulation
Mitrano, P. P., LaGrassa, A., Kroemer, O., & Berenson, D. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161366 · Cited by 23
Takagi-Sugeno-Kang Fuzzy System Towards Label-scarce Incomplete Multi-View Data Classification
Zhang, W., Deng, Z., Lou, Q., Zhang, T., Choi, K., & Wang, S. (2023) · Information Sciences · doi:10.1016/j.ins.2023.119466 · Cited by 18
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.