Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 1,301–1,400 of 5,239
Belinkov, Y. (2018) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 34
An Explicitly Relational Neural Network ArchitecturePreprint
Shanahan, M., Nikiforou, K., Creswell, A., Kaplanis, C., Barrett, D. G. T., & Garnelo, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10307 · Cited by 53
“Curricular pre-training has a decade-long pedigree in deep learning (Bengio et al., 2009).”
BackgroundNeural Execution of Graph AlgorithmsPreprint
Veličković, P., Ying, R., Padovano, M., Hadsell, R. T., & Blundell, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.10593 · Cited by 195
“Lastly, we report the performance of a curriculum learning (Bengio et al., 2009) strategy (as curriculum): here, BFS is learnt first in isolation (to perfect validation accuracy), followed by finetuning on Bellman-Ford.”
MethodNeural Language Generation: Formulation, Methods, and Evaluation
Gârbacea, C., & Mei, Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.15780 · Cited by 24
“using mostly the generated token) based on curriculum learning (Bengio et al., 2009).”
MethodAbstractive Summarization of Spoken and Written Instructions with BERT
Savelieva, A., Au-Yeung, B., & Ramani, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.09676 · Cited by 35
“The idea of applying curriculum learning [1] in natural language processing has been a growing topic of interest [26].”
BackgroundBANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale PretrainingPreprint
Qi, W., Gong, Y., Jiao, J., Yan, Y., Chen, W., Liu, D., Tang, K., Li, H., Chen, J., Zhang, R., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15525 · Cited by 32
“Curriculum learning (Bengio et al., 2009) is used to benefit NAR generation from AR generation such as Guo et al. (2020a).”
Learning to Iteratively Solve Routing Problems with Dual-Aspect Collaborative TransformerPreprint
Ma, Y., Li, J., Cao, Z., Song, W. W., Zhang, L., Chen, Z., & Tang, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02544 · Cited by 181
“In doing so, 1) it increases the probability for the agent to observe better solutions and thus reduce the variance of the value function; 2) it increases the difficulty of the learning task (higher-quality solutions are harder to improve) in a gradual manner and achieves better sample efficiency [34].”
BackgroundLesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., & Díaz-Rodríguez, N. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.004 · Cited by 506
“Curriculum Learning Curriculum learning (Bengio et al., 2009) is a training process that proposes a sequence of more and more difficult tasks to a learning algorithm in order to make it able to learn, at last, a generally harder task.”
BackgroundSecure federated learning applied to medical imaging with fully homomorphic encryption
Lessage, X., Collier, L., Ouytsel, C.-H. B. V., Legay, A., Mahmoudi, S., & Massonet, P. (2024) · IEEE International Conference on AI in Cybersecurity (ICAIC) · doi:10.1109/icaic60265.2024.10433836 · Cited by 34
“Similarly, Jimenze-Sanchez et al. [30] used mammography and curriculum learning [9] in their FL architecture to improve the consistency of their local models.”
MethodGuo, H., Somayajula, S. A., Hosseini, R., & Xie, P. (2024) · Scientific Reports · doi:10.1038/s41598-024-53955-8 · Cited by 29
“Different from the classical curriculum learning setting where the training samples are ordered and trained in a easy to difficult fashion defined by the training loss 61 , our method defines difficulty in the data pre-processing stage.”
MethodCL-MAE: Curriculum-Learned Masked Autoencoders
Madan, N., Ristea, N.-C., Nasrollahi, K., Moeslund, T. B., & Ionescu, R. T. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00248 · Cited by 52
“…curriculum involves using either approaches based on ex-ternal complexity measures, such as the degree of occlusion and the complexity of the shape [3, 16], or self-paced learning techniques [22,27,29,52], in which the neural network dynamically assesses the difficulty of training samples based on…”
BackgroundMethodInfluentialA Survey on Self-Supervised Monocular Depth Estimation Based on Deep Neural Networks
Dong, Q., Zhou, Z., Qiu, X., & Zhang, L. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3552598 · Cited by 30
“StrAT [105] further introduced the curriculum learning technique [106] into the adversarial framework, which gradually increased the distance of the stereo-baseline at the training stage for training the network from easy tasks to harder ones.”
Deistler, M., Kadhim, K. L., Pals, M., Beck, J., Huang, Z., Gloeckler, M., Lappalainen, J. K., Schröder, C., Berens, P., Gonçalves, P. J., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02895-w · Cited by 59
A comprehensive survey on imbalanced data learning
Gao, X., Xie, D., Zhang, Y., Wang, Z., Chen, C., He, C., Yin, H., & Zhang, W. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-025-50274-7 · Cited by 83
“3) Curriculum learning: Curriculum learning [168] is the strategy of learning from easy to hard, which can significantly improve the generalization of the deep model.”
Neural Compositional Denotational Semantics for Question Answering
Gupta, N., & Lewis, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1239 · Cited by 19
“Training Details: Training the model is challenging since it needs to learn both good syntactic structures and the complex semantics of neural modules—so we use Curriculum Learning (Bengio et al., 2009) to pre-train the model on an easier sub-set of questions.”
MethodLearning to Teach in Cooperative Multiagent Reinforcement Learning
Omidshafiei, S., Kim, D.-K., Liu, M., Tesauro, G., Riemer, M., Amato, C., Campbell, M., & How, J. P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016128 · Cited by 96
“Works on curriculum learning (Bengio et al. 2009) are also related, particularly automated curriculum learning (Graves et al. 2017).”
BackgroundCollaborative representation with curriculum classifier boosting for unsupervised domain adaptation
Han, C., Zhou, D., Xie, Y., Gong, M., Lei, Y., & Shi, J. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2020.107802 · Cited by 24
Gao, Z., Shi, J., Zhang, X., Li, Y., Zhang, H., Wu, J., Wang, C., Meng, D., & Li, C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87237-3_13 · Cited by 35
“This insight is similar to curriculum learning [11] which has been proven to be effective in histopathology image [12].”
BackgroundNon-Linear Fusion for Self-Paced Multi-View Clustering
Huang, Z., Ren, Y., Pu, X., et al. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475471 · Cited by 23
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398 · Cited by 31
Uncertainty Estimation and Reduction of Pre-trained Models for Text Regression
Wang, Y., Beck, D., Baldwin, T., & Verspoor, K. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00483 · Cited by 23
“Our approach also has a similar flavor to self-paced curricular learning (Bengio et al., 2009; Kumar et al., 2010; Wan et al., 2020), in which the aim is to choose ‘‘hard’’ examples and gradually increase the difficulty of learning content, differing from the criteria in our setting— ‘‘clean’’ ones.”
BackgroundState Dropout-Based Curriculum Reinforcement Learning for Self-Driving at Unsignalized Intersections
Khaitan, S., & Dolan, J. M. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981109 · Cited by 34
“Apart from accelerating the training process, it has also been proposed that using a curriculum can help find better local minima as compared to training without curriculum [7].”
BackgroundMethodFan, F., Xu, G.-L., Feng, N., et al. (2023) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2023.06.011 · Cited by 23
Schmoigl-Tonis, M., Schranz, C., & Müller-Putz, G. (2023) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2023.1251690 · Cited by 29
Pred-NBV: Prediction-Guided Next-Best-View Planning for 3D Object Reconstruction
Dhami, H., Sharma, V. D., & Tokekar, P. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341650 · Cited by 44
“To improve predictions, we fine-tune PoinTr using a curriculum framework, which dictates training the network over easy to hard tasks by increasing the difficulty in steps during learning [19].”
BackgroundMethodEfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training
Wang, Y., Yue, Y., Lu, R., Han, Y., Song, S., & Huang, G. M. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3401036 · Cited by 38
“Curriculum learning is a training paradigm inspired by the organized learning order of examples in human curricula [8], [25], [26].”
BackgroundResultInfluentialDeep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices
Małkiński, M., & Mańdziuk, J. (2025) · ACM Computing Surveys · doi:10.1145/3715093 · Cited by 61
Optimizing Instructional Policies
Lindsey, R., Mozer, M. C., Huggins, W. J., & Pashler, H. (2013) · Cited by 23
On Studying Human Teaching Behavior with Robots: a Review
Vollmer, A.-L., & Schillingmann, L. (2017) · Review of Philosophy and Psychology · doi:10.1007/s13164-017-0353-4 · Cited by 26
“…often by participants is described as starting from extreme (i.e., ideal) examples and then move toward the decision boundary which the authors state to be evidence of the ‘curriculum learning principle’ (Bengio et al. 2009) and which reflects the same strategy found in human-human interaction.”
BackgroundSelf-paced data augmentation for training neural networks
Takase, T., Karakida, R., & Asoh, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.02.080 · Cited by 23
“Here, we discuss why the proposed SPA method performed well in consideration of two well-established training strategies, i.e., curriculum learning [7] and desirable changes to loss function instability [25].”
BackgroundMethodLearning with Memory-based Virtual Classes for Deep Metric Learning
Ko, B., Gu, G., & Kim, H.-G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01158 · Cited by 31
“Imposing CL for model training has been shown to accelerate and improve the training process in many machine learning tasks [1, 47, 13, 18].”
BackgroundDynamic-weighting hierarchical segmentation network for medical images
Guo, X., Yang, C., & Yuan, Y. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102196 · Cited by 23
Three Dependency-and-Boundary Models for Grammar Induction
Spitkovsky, V. I., Alshawi, H., & Jurafsky, D. S. (2012) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 26
Backplay: "Man muss immer umkehren"Preprint
Resnick, C., Răileanu, R., Kapoor, S., Peysakhovich, A., Cho, K., & Bruna, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06919 · Cited by 59
“In contrast, Backplay only requires access to a single trajectory of states visited by this policy and uses these states to initialize a curriculum (Bengio et al., 2009) rather than as a supervised objective.”
Discriminative Neural Clustering for Speaker DiarisationPreprint
Li, Q., Kreyssig, F., Zhang, C., & Woodland, P. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.09703 · Cited by 20
“A curriculum learning (CL) [39, 40] approach, i.”
Sensing flow gradients is necessary for learning autonomous underwater navigation
Jiao, Y., Hang, H., Merel, J., & Kanso, E. (2025) · Nature Communications · doi:10.1038/s41467-025-58125-6 · Cited by 23
Li, Z., Zeng, C., Deng, Z., Xu, Q., He, B., & Zhang, J. (2024) · IEEE Systems Man and Cybernetics Magazine · doi:10.1109/msmc.2022.3231416 · Cited by 28
Reinforcement Learning-Based Bucket Filling for Autonomous Excavation
Egli, P., Terenzi, L., & Hutter, M. (2024) · IEEE transactions on field robotics. · doi:10.1109/tfr.2024.3432508 · Cited by 31
“Curriculum learning [6] has established itself to train complex RL tasks.”
BackgroundWhen Meta-Learning Meets Online and Continual Learning: A Survey
Son, J., Lee, S., & Kim, G. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3463709 · Cited by 49
“This invariance can be beneficial in some scenarios, but it can also be a limitation when the order of the data is crucial, such as curriculum learning [132] or reinforcement learning [133].”
LLM-R 2 : A Large Language Model Enhanced Rule-Based Rewrite System for Boosting Query Efficiency
Li, Z., Yuan, H., Wang, H., Cong, G., & Bing, L. (2024) · Proceedings of the VLDB Endowment · doi:10.14778/3696435.3696440 · Cited by 86
“In addition, to address the challenge of limited training data, we propose to utilize the learning curriculum technique [8] to schedule the training data from easy to hard.”
Cuoco, E., Cavaglià, M., Heng, I. S., Keitel, D., & Messenger, C. (2025) · Living Reviews in Relativity · doi:10.1007/s41114-024-00055-8 · Cited by 53
Learning Category-Specific 3D Shape Models from Weakly Labeled 2D Images
Zhang, D., Han, J., Yang, Y., & Huang, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.382 · Cited by 21
“This is closely related to the field of curriculum learning, which was originally proposed in [5] and has been successfully used in other applications like object detection and recognition [8, 38].”
BackgroundDuminy, N., Nguyen, S. M., & Duhaut, D. (2019) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00087 · Cited by 23
“To address the challenge of multi-task motor learning, we will take the point of view of continual learning, also named life-long or curriculum Bengio et al. (2009) learning, that constructs a sophisticated understanding of the world from its own experience to apply previously learned knowledge and skills to new situa…”
BackgroundInfluentialGong, T., Wang, G., Ye, J., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11698 · Cited by 22
“Recently, the iterative multi-stage training strategy enjoys increasing popularity in curriculum learning (CL) (Bengio et al. 2009) and self-paced learning (SPL) (Kumar, Packer, and Koller 2010).”
BackgroundSequence Modeling in Unsupervised Single-Channel Overlapped Speech Recognition
Chen, Z., & Droppo, J. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461939 · Cited by 20
“The strategy is motivated by the curriculum learning theory [12], which integrates both modularization and joint training.”
BackgroundSelf-paced stacked denoising autoencoders based on differential evolution for change detection
Li, H., Gong, M., Wang, C., & Miao, Q. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.07.021 · Cited by 20
Transparent Machine Education of Neural Networks for Swarm Shepherding Using Curriculum Design
Gee, A. G., & Abbass, H. A. (2019) · doi:10.1109/ijcnn.2019.8852209 · Cited by 22
“[3] were then able to show significant improvements in speed of convergence to local minima, as well as the quality of the local minima obtained, by implementing ‘Curriculum Learning’ which is the concept describing the training strategy of organising concepts from simple to more complex ones before presenting them to…”
BackgroundMulti-view based integrative analysis of gene expression data for identifying biomarkers
Yang, Z., Liu, X., Shu, J., Zhang, H., Ren, Y., Xu, Z., & Liang, Y. Q. (2019) · Scientific Reports · doi:10.1038/s41598-019-49967-4 · Cited by 20
“It was proposed based on the core idea of curriculum learning 34 .”
BackgroundProgressFace: Scale-Aware Progressive Learning for Face Detection
Zhu, J., Li, D., Han, T., Tian, L., & Shan, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58539-6_21 · Cited by 24
“Inspired by curriculum learning [1], we propose a progressive training mechanism for face detection by gradually incorporating large-to-small samples.”
BackgroundMethodInfluentialNguyen, T., Luu, T. M., Pham, T. X., Rakhimkul, S., & Yoo, C. D. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413446 · Cited by 31
“This strategy is consistent with curriculum learning which proposes training on tasks that are not too easy and also not too hard [22].”
BackgroundModel independent analysis of coupled-channel scattering: A deep learning approach
Sombillo, D. L. B., Ikeda, Y., Sato, T., et al. (2021) · Physical review. D/Physical review. D. · doi:10.1103/physrevd.104.036001 · Cited by 22
Learning Algebraic Recombination for Compositional Generalization
Liu, C., An, S., Lin, Z., Liu, Q., Chen, B., Lou, J., Wen, L., Zheng, N., & Zhang, D. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.97 · Cited by 45
“To help the model converge better, we use a simple curriculum learning (Bengio et al., 2009) strategy to train the model.”
MethodBayesian controller fusion: Leveraging control priors in deep reinforcement learning for robotics
Rana, K., Dasagi, V., Haviland, J., Talbot, B., Milford, M. J., & Sünderhauf, N. (2023) · The International Journal of Robotics Research · doi:10.1177/02783649231167210 · Cited by 50
From Superficial to Deep: Language Bias driven Curriculum Learning for Visual Question Answering
Lao, M., Guo, Y., Liu, Y., Chen, W., Pu, N., & Lew, M. S. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475492 · Cited by 34
“[7] rst propose the concept of curriculum learning (CL), which formalizes an easy-to-hard strategy for training machine learning models inspired by human brains.”
BackgroundMethodSemi-Autoregressive Image Captioning
Yan, X., Fei, Z., Li, Z., Wang, S., Huang, Q., & Tian, Q. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475179 · Cited by 29
Wang, Z., Xuan, J., & Shi, T. (2022) · Advanced Engineering Informatics · doi:10.1016/j.aei.2022.101773 · Cited by 25
BioSLAM: A Bioinspired Lifelong Memory System for General Place Recognition
Yin, P., Abuduweili, A., Zhao, S., Xu, L., Liu, C., & Scherer, S. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3306615 · Cited by 38
“Inspired by animal training [63] and curriculum learning [64], it is practical to differentiate the data samples into different difficulty levels, such as “easy,” “medium,” and “hard.””
BackgroundPath Spuriousness-aware Reinforcement Learning for Multi-Hop Knowledge Graph Reasoning
Jiang, C., Zhu, T., Zhou, H., Liu, C., Deng, T., Hu, C., & Li, J. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.232 · Cited by 24
“This equation is inspired by curriculum learning, which sets different optimizing goals in different learning stages (Bengio et al., 2009).”
BackgroundRPAU: Fooling the Eyes of UAVs via Physical Adversarial Patches
Liu, T., Yang, C., Liu, X., Han, R., & Ma, J. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3317054 · Cited by 26
“Inspired by curriculum learning [35], we have introduced a time-dependent mechanism to balance the weights of different variables of the objective function.”
BackgroundMethodInfluentialWang, J., Liang, J., Xiao, Y., Zhou, J. T., Fang, Z., & Yang, F. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3352984 · Cited by 26
“Finally, inspiredbytheefficacyofgraduallydifficulty-levellearning [18] in various computer vision fields [19], [20], [21], we develop an easy-to-hard learning strategy to direct the model’s attention to incrementally difficulty-level regions throughout the training phase of breast lesion segmentation.”
BackgroundMethodDomain Generalization via Balancing Training Difficulty and Model Capability
Jiang, X., Huang, J., Jin, S., & Lu, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01741 · Cited by 29
“Learning Strategies.”
BackgroundOn the utility of curricula in unsupervised learning of probabilistic grammars
Tu, K., & Honavar, V. (2011) · doi:10.5591/978-1-57735-516-8/ijcai11-256 · Cited by 22
Paper Abstract Writing through Editing Mechanism
Wang, Q., Zhou, Z., Huang, L., Whitehead, S., Zhang, B., Ji, H., & Knight, K. K. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2042 · Cited by 26
“We de-sign a Writing-editing Network which shares ideas with Curriculum Learning (Bengio et al., 2009), where training on a data point from coarse to fine-grained can lead to better convergence (Krueger and Dayan, 2009).”
Liu, D., Fu, J., Qu, Q., & Lv, J. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2943018 · Cited by 42
“In order to gradually teach the model to produce stable sentences, we provide more and more difficult lexical constraints, which is a form of curriculum learning (Bengio et al., 2009).”
BackgroundAdult Image Classification by a Local-Context Aware Network
Wang, X., Cheng, F., Wang, S., Sun, H., Liu, G., & Zhou, C. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451366 · Cited by 23
“Thirdly, a curriculum learning strategy [6][7] is proposed to facilitate retraining the LocoaNet on larger datasets.”
MethodAda-boundary: accelerating DNN training via adaptive boundary batch selection
Song, H., Kim, S., Kim, M., & Lee, J.-G. (2020) · Machine Learning · doi:10.1007/s10994-020-05903-6 · Cited by 22
“Curriculum learning (Bengio et al. 2009), inspired by the perceived way that humans and animals learn, first takes easy samples and then gradually increases the difficulty of samples using a manual method.”
MethodGenerative Adversarial Image Synthesis with Decision Tree Latent Controller
Kaneko, T., Hiramatsu, K., & Kashino, K. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00691 · Cited by 23
“To mitigate these problems, we propose a hierarchical conditional mutual information regularization (HCMI), which is an extension of MI [4] and conditional MI (CMI) [15] to hierarchical conditional settings and optimized it with a newly defined curriculum learning [3] method that we also propose.”
MethodMulti-modal Discriminative Model for Vision-and-Language Navigation
Huang, H., Jain, V., Mehta, H., Baldridge, J., & Ie, E. (2019) · Proceedings of the Combined Workshop on Spatial Language Understanding ( · doi:10.18653/v1/w19-1605 · Cited by 22
“To further confirm this hypothesis, we train a discriminator using curriculum learning (Bengio et al., 2009) where the model is first trained on only PS negatives and then fine-tuned on PR and RW negatives.”
MethodA gated piecewise CNN with entity-aware enhancement for distantly supervised relation extraction
Wen, H., Zhu, X., Zhang, L., & Li, F. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102373 · Cited by 22
With Whom to Communicate: Learning Efficient Communication for Multi-Robot Collision Avoidance
Serra-Gómez, Á., Brito, B., Zhu, H., Chung, J. J., & Alonso–Mora, J. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341762 · Cited by 23
“Hence, we employ curriculum learning [30], training the agents first in a simple scenario, where no communication is needed, and continuously introducing more difficult and complex scenarios where the agents must learn when to communicate.”
MethodUnsupervised Dense Shape Correspondence using Heat Kernels
Aygün, M. M., Lähner, Z., & Cremers, D. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00067 · Cited by 26
“This learning strategy is used for training deep neural networks and showed effective performance on various problems [20, 5, 14].”
MethodNavigating the landscape of multiplayer games
Omidshafiei, S., Tuyls, K., Czarnecki, W. M., Santos, F. C., Rowland, M., Connor, J., Hennes, D., Muller, P., Pérolat, J., Vylder, B. D., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-19244-4 · Cited by 51
“Understanding and decomposing the characterizing features of games can be leveraged for downstream training of agents via curriculum learning 14 , which seeks to enable agents to learn increasingly-complex tasks.”
BackgroundCooperative Object Transportation Using Curriculum-Based Deep Reinforcement Learning
Eoh, G., & Park, T.-H. (2021) · Sensors · doi:10.3390/s21144780 · Cited by 25
Morphological Development in Robotic Learning: A Survey
Naya-Varela, M., Faíña, A., & Duro, R. J. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3052548 · Cited by 28
EdgeStereo: A Context Integrated Residual Pyramid Network for Stereo MatchingPreprint
Song, X., Zhao, X., Hu, H., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.05196 · Cited by 22
“The decoder is formulated as a residual pyramid inspired by curriculum and residual learning [3,14].”
MethodGuided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied AgentsPreprint
Yu, H., Lian, X., Zhang, H., & Xu, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08329 · Cited by 24
Kung, H. T., McDanel, B., & Zhang, S. Q. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.04770 · Cited by 25
“Our model pruning and retraining method can be view as part of a larger area of research shared by teacher-student networks [50, 58] and curriculum learning [8].”
MethodSelf-supervised Video Representation Learning by Uncovering Spatio-temporal Statistics
Wang, J., Jiao, J., Bao, L., He, S., Liu, W., & Liu, Y. (2020) · PubMed · doi:10.1109/tpami.2021.3057833 · Cited by 65
“[55] in 2009 and the key concept is to present the network with more difficult samples gradually.”
BackgroundZombori, Z., Csiszárik, A., Michalewski, H., Kaliszyk, C., & Urban, J. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-86059-2_10 · Cited by 22
“tice such principles as that nine times a number is the same as ten times the number minus the number ...”. The idea of learning using a curriculum of problems is also widely used in machine learning [20, 9, 60, 64] and in this work we apply curriculum learning to automatic theorem proving focusing on arithmetic. Our…”
MethodS2-Transformer for Mask-Aware Hyperspectral Image Reconstruction
Wang, J., Li, K., Zhang, Y., Yuan, X., & Tao, Z. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3543842 · Cited by 50
Ding, R., Lu, H. J., & Liu, M. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3551514 · Cited by 49
“Experiments test the impacts of [30], C URRICULUM L EARNING (CL) [38], G RAD -”
Face Anonymization Made Simple
Kung, H.-W., Varanka, T., Saha, S., Sim, T., & Sebe, N. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00110 · Cited by 51
“To enhance performance and generalization, we adopted strategies from curriculum learning [5].”
Exploring Implicit Feedback for Open Domain Conversation Generation
Zhang, W., Li, L., Cao, D., & Liu, T. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11253 · Cited by 19
“For training the Seq2Seq model, we use the curriculum learning approach (Bengio et al. 2009) to generate each utterance.”
MethodCurriculum goal masking for continuous deep reinforcement learning
Eppe, M., Magg, S., & Wermter, S. (2019) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2019.8850721 · Cited by 27
“[2]), where the learner does not attempt to solve a hard problem immediately using a bruteforce trial-and-error strategy but instead attempts to follow a certain curriculum that involves a favorable sequence of subgoals within the learning process.”
BackgroundMeasuring Human Perception to Improve Handwritten Document Transcription
Grieggs, S., Shen, B., Rauch, G., Li, P., Ma, J., Chiang, D., Price, B. L., & Scheirer, W. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3092688 · Cited by 25
“physical measurements in a curriculum learning regime [56], which also considers example difficulty.”
BackgroundArtificial Intelligence – The Big Picture
Abele, D., & D’Onofrio, S. (2020) · Springer eBooks · doi:10.1007/978-3-658-27941-7_2 · Cited by 33
A conceptual framework for externally-influenced agents: an assisted reinforcement learning review
Bignold, A., Cruz, F., Taylor, M. E., Brys, T., Dazeley, R., Vamplew, P. W., & Foale, C. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03489-y · Cited by 45
“Third, TL could be a critical step within a curriculum learning approach (Taylor 2009; Bengio et al. 2009).”
BackgroundAudio Tagging by Cross Filtering Noisy Labels
Zhu, B., Xu, K., Kong, Q., Wang, H., & Peng, Y. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3008832 · Cited by 29
“Inspired by the curriculum learning [38], [39], we gradually increase the number of pseudo curated data selected from N in each epoch.”
BackgroundMethodOnline Disease Diagnosis with Inductive Heterogeneous Graph Convolutional Networks
Wang, Z., Wen, R., Chen, X., Cao, S., Huang, S., Qian, B., & Zheng, Y. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449795 · Cited by 47
“nto probability Conference’17, July 2017, Washington, DC, USA Zifeng Wang1, Rui Wen2, Xi Chen2, Shilei Cao2, Shao-Lun Huang1, Buyue Qian3, Yefeng Zheng2 distribution pi = softmax(hi)= eh i ˝ j e h j ∈[0,1]. (10) For example, if the target number of retrieved symptoms is k, we will pick k ×pi related symptoms from N(di…”
BackgroundLiu, Y., Guo, B., Zhang, D., et al. (2021) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3447271 · Cited by 21
Efficient conditional GAN transfer with knowledge propagation across classes
Shahbazi, M., Huang, Z., Huang, Paudel, D. P., Chhatkuli, A., & VAN, G. L. (2021) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 22
What is Next when Sequential Prediction Meets Implicitly Hard Interaction?
Hu, K., Li, L., Xie, Q., Liu, J., & Tao, X. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482492 · Cited by 28
“It is harmful to make model fit them, which may hurt the generalization [3, 13].”
BackgroundSAT: Improving Adversarial Training via Curriculum-Based Loss Smoothing
Sitawarin, C., Chakraborty, S., & Wagner, D. (2021) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3474369.3486878 · Cited by 71
“To this end, we observe that the concept of curriculum learning [3], which advocates that model training be initiated with “easy” samples before introducing the “hard” ones, can naturally help overcome the above problems.”
BackgroundLuciw, N. J., Shirzadi, Z., Black, S. E., Goubran, M., & MacIntosh, B. J. (2022) · Magnetic Resonance in Medicine · doi:10.1002/mrm.29193 · Cited by 22
“27 We removed inputs in a systematic outside-in fashion: first the 100 ms PLD image was removed, then 2100 ms, then 500 ms. Curriculum training was performed for 600 total epochs wherein the inputs with missing PLDs were introduced starting at epoch 200, after which the difficulty of training…”
MethodA Survey on Dropout Methods and Experimental Verification in Recommendation
Li, Y., Ma, W., Chen, C., Zhang, M., Liu, Y., Ma, S., & Yang, Y. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3187013 · Cited by 38
“Inspired by curriculum learning [93], in contrast to Annealed Dropout [34], Curriculum Dropout increases dropout ratio as the number of training epochs increases.”
MethodChe, H., Jin, H., & Chen, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16437-8_50 · Cited by 25
“Such a dynamic and adaptive curriculum helps models build robust feature representations and speed up training [19].”
BackgroundInfluentialUniversal Image Steganalysis Based on Convolutional Neural Network with Global Covariance Pooling
Deng, X., Chen, B., Luo, W., & Luo, D. (2022) · Journal of Computer Science and Technology · doi:10.1007/s11390-021-0572-0 · Cited by 25
“For the remaining payloads and the quality factor of 75, we adopt the curriculum training strategy [31] , that is, we initialize the network with the network trained for a higher payload.”
MethodLearning to Anticipate Future with Dynamic Context Removal
Xu, X., Li, Y., & Lu, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01240 · Cited by 31
DRL-based Path Planner and its Application in Real Quadrotor with LIDAR
Yang, Y., Hou, Z., Chen, H., & Lu, P. (2023) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-023-01819-0 · Cited by 22
Collaborative Learning-Based Scheduling for Kubernetes-Oriented Edge-Cloud Network
Shen, S., Han, Y., Wang, X., Wang, S., & Leung, V. C. M. (2023) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2023.3267168 · Cited by 31
“Inspired by curriculum learning proposed in [36], we first use simple and short service request sequences for training, and then moderately introduce more sophisticated request sequences step by step, so that the scheduling policy can be gradually improved.”
Method
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.