Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 2,001–2,100 of 5,240
MC-BERT: Efficient Language Pre-Training via a Meta ControllerPreprint
Xu, Z., Gong, L., Ke, G., He, D., Zheng, S., Wang, L., Bian, J., & Liu, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05744 · Cited by 27
“Our method is related to curriculum learning [2].”
MethodModel-Based Deep Reinforcement Learning for High-Dimensional Problems, a Survey.Preprint
Plaat, A., Kosters, W. A., & Preuß, M. (2020) · arXiv (Cornell University) · Cited by 13
Evaluating Curriculum Learning Strategies in Neural Combinatorial OptimizationPreprint
Lisicki, M., Afkanpour, A., & Taylor, G. W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06188 · Cited by 27
“Curriculum learning (CL) is based on the premise that neural networks learn faster from tasks gradually increasing in complexity [4, 9].”
BackgroundCIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature SelectionPreprint
Yan, H., Zhang, J., Niu, G., Feng, J., Tan, V. Y. F., & Sugiyama, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.05311 · Cited by 50
“Besides, inspired by curriculum learning (Cai et al., 2018; Bengio et al., 2009), Zhang et al. (2020) proposed FAT to train models with increasingly adversarial data, which enhances generalization without sacrificing robustness.”
HonestLLM: Toward an Honest and Helpful Large Language Model
Gao, C., Wu, S., Huang, Y., Chen, D., Zhang, Q., Fu, Z., Wan, Y., Sun, L., & Zhang, X. (2024) · arXiv · doi:10.48550/arxiv.2406.00380 · Cited by 11
Deep Learning in Network-Level Performance Prediction Using Cross-Layer Information
Cao, Q., Pun, M.-O., & Yi, C. (2022) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2022.3163274 · Cited by 13
“We construct this optimization problem based on the studies [37] and [38] where the authors claim that data should be weighted according to their importance for model training, which could provide better generalization to the neural network model.”
MethodCurriculum-NAS: Curriculum Weight-Sharing Neural Architecture Search
Zhou, Y., Wang, X., Chen, H., Duan, X., Guan, C., & Zhu, W. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548271 · Cited by 15
“Curriculum learning (CL) [2, 51] is a strategy of training from ease, imitating the procedure of human learning with curricula.”
BackgroundSelfCCL: Curriculum Contrastive Learning by Transferring Self-Taught Knowledge for Fine-Tuning BERT
Dehghan, S., & Amasyalı, M. F. (2023) · Applied Sciences · doi:10.3390/app13031913 · Cited by 13
Real-Time Motion Analysis With 4D Deep Learning for Ultrasound-Guided Radiotherapy
Bengs, M., Sprenger, J., Gerlach, S., et al. (2023) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2023.3262422 · Cited by 10
“fully addressed with our training approach, which combines curriculum learning [36] and TBTT [35].”
BackgroundLi, Z., Yao, Y., Quan, Z., Qi, L., Feng, Z., & Yang, W. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3343878 · Cited by 12
“Curriculum Example Mining: Inspired by the above investigation, we design a curriculum-learning-based sampling scheme [49], namely curriculum example mining (CEM).”
MethodDisentangled Representation Learning for Controllable Person Image Generation
Xu, W., Long, C., Nie, Y., & Wang, G. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3345180 · Cited by 15
“Motivated by human’s easy-to-hard learning process, we adopt the curriculum learning strategy [14] and start from a relatively easy step, then gradually increase the difficulty levels to the complete mask-agnostic step.”
BackgroundMethodGRA: Graph Representation Alignment for Semi-Supervised Action Recognition
Huang, K.-H., Huang, Y.-B., Lin, Y.-X., Hua, K., Tanveer, M., Lu, X., & Razzak, I. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3347593 · Cited by 11
“Another work [30] approached confirmation bias by redesigning the retraining process with curriculum (easy-to-hard) concept [49].”
BackgroundLate Stopping: Avoiding Confidently Learning from Mislabeled Examples
Yuan, S., Feng, L., & Liu, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01473 · Cited by 28
“At the low level (Steps 4-8 of Algorithm 1), the sample selection strategy follows a “from easy to hard” curriculum learning procedure [5, 18].”
BackgroundRuano, J., Zuleta, M. A. G., Romero, E., & Manzanera, A. (2024) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2024.102390 · Cited by 11
“…(CL) is to ensure that the found solution follows the global extremum by gradually complexifying the parameter space while improving generalization and speeding up convergence (Parisi et al., 2019; Bengio et al., 2009; Narvekar et al., 2020; Graves et al., 2017; Hacohen and Weinshall, 2019).”
Co-Evolving Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representation
Siddiqua, A., Liu, S., Nipu, A. S., Harris, A., & Liu, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3430037 · Cited by 14
Knowledge transfer in lifelong machine learning: a systematic literature review
Khodaee, P., Viktor, H. L., & Michalowski, W. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10853-9 · Cited by 22
Self-paced contrastive learning for knowledge tracing
Dai, H. L., Yun, Y., Zhang, Y., An, R., Zhang, W., & Shang, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128366 · Cited by 14
Robust speech command recognition in challenging industrial environments
Bini, S., Carletti, V., Saggese, A., & Vento, M. (2024) · Computer Communications · doi:10.1016/j.comcom.2024.107938 · Cited by 11
Accelerating Laboratory Automation Through Robot Skill Learning For Sample Scraping*
Pizzuto, G., Wang, H., Fakhruldeen, H., Peng, B., Luck, K. S., & Cooper, A. I. (2024) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case59546.2024.10711291 · Cited by 25
Text-prompt Camouflaged Instance Segmentation with Graduated Camouflage Learning
He, Z., Xia, C., Qiao, S., & Li, J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681132 · Cited by 16
“This insight has been supported by previous research [3], highlighting the need for a more sophisticated training approach.”
InfluentialAdvancing 3D Object Grounding Beyond a Single 3D Scene
Huang, W., Liu, D., & Hu, W. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680758 · Cited by 18
Learning Latent Wireless Dynamics From Channel State Information
Chaaya, C. B., Girgis, A. M., & Bennis, M. (2024) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2024.3510943 · Cited by 29
“Inspired by the ML literature on curriculum learning [13], we propose a simple two-stage training method.”
Progressive Learning Strategy for Few-Shot Class-Incremental Learning
Hu, K., Wang, Y., Zhang, Y., & Gao, X. (2025) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2025.3525724 · Cited by 13
“First, inspired by the concept of curriculum learning [10], we observed that human learning is a gradual and progressive process.”
Seed: Bridging Sequence and Diffusion Models for Road Trajectory Generation
Rao, X., Shang, S., Jiang, R., et al. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714951 · Cited by 10
“To address this issue, we adopt a curriculum learning paradigm [1], which suggests that in the early stages of training, models are more sensitive to noise and difficult samples, and thus can benefit from a staged training approach.”
Context-aware data augmentation for enhanced speech command recognition in industrial environments
Simone, G. D., Greco, A., Rosa, F., Saggese, A., & Vento, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-01886-3 · Cited by 13
“…then, we augmented these noisy-free samples with samples containing industrial noise (such as screwdrivers) and generic noise (such as people talking in the background), leveraging Curriculum-Learning 17 procedure to dynamically adapt the Signal-to-Noise Ratio (SNR) during the learning procedure.”
Spangher, L., Bonotto, M., Arnold, W., Chayapathy, D., Gallingani, T., Spangher, A., Cannarile, F., Bigoni, D., Marchi, E. D., & Rea, C. (2025) · Journal of Fusion Energy · doi:10.1007/s10894-025-00495-2 · Cited by 16
Jafari, R., Sarhadi, P., Paykani, A., Refaat, S. S., & Asef, P. (2025) · IEEE Access · doi:10.1109/access.2025.3587938 · Cited by 14
“The original concept of curriculum learning was first introduced in [47], an effective strategy in RL that facilitates stable and efficient policy training by introducing tasks with increasing complexity in a progressive manner [48].”
Saito, A. (2018) · doi:10.18653/v1/w18-5707 · Cited by 16
“Curriculum Learning The first proposition of the concept of curriculum learning is in (Bengio et al., 2009).”
BackgroundTikhonov, E. E., Sosin, A. I., & Evdokimov, A. A. (2018) · International Multi-Conference on Industrial Engineering and Modern Technologies (FarEastCon) · doi:10.1109/fareastcon.2018.8602679 · Cited by 10
Multi-View Attention Network for Visual Dialog
Park, S., Whang, T., Yoon, Y. C., & Lim, H. (2021) · Applied Sciences · doi:10.3390/app11073009 · Cited by 28
“). Also, Qi et al. (2020) proposed causal intervention algorithms that can be applicable to other visual dialog models and Agarwal et al. (2020) proposed curriculum fine-tuning inspired by the work of Bengio et al. (2009). Model In the visual dialog task (Das et al. 2017), a dialog agent is given a set of multimodal in…”
BackgroundTowards Reasonably-Sized Character-Level Transformer NMT by Finetuning Subword Systems
Libovický, J., & Fraser, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.203 · Cited by 11
“We test two methods for finetuning subword models to reach character-level models: first, direct finetuning of subword models, and second, iteratively removing BPE merges in several steps in a curriculum learning setup (Bengio et al., 2009).”
MethodDeep Reinforcement Learning Applied to IEEE Very Small Size Soccer Strategy
de Medeiros, T. F., Máximo, M. R. O. A., & Yoneyama, T. (2020) · Latin American Robotics Symposium, Brazilian Symposium on Robotics and Workshop on Robotics in Education (LARS/SBR/WRE) · doi:10.1109/lars/sbr/wre51543.2020.9306954 · Cited by 12
Scale-Aware Anchor-Free Object Detection via Curriculum Learning for Remote Sensing Images
Cai, W., Zhang, B., & Wang, B. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3115796 · Cited by 10
Progressive Diversified Augmentation for General Robustness of DNNs: A Unified Approach
Yu, H., Liu, A., Li, G., Yang, J., & Zhang, C. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3121150 · Cited by 12
Arbitrary-Sized JPEG Steganalysis Based on Fully Convolutional Network
Su, A., Zhao, X., & He, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-95398-0_14 · Cited by 11
Niwa, T., Shibata, K., & Jimbo, T. (2022) · Springer proceedings in advanced robotics · doi:10.1007/978-3-030-92790-5_16 · Cited by 14
Curriculum Incremental Deep Learning on BreakHis DataSet
Mayouf, M. S., & de Saint-Cyr, F. D. (2022) · International Conference on Computer Technology Applications (ICCTA) · doi:10.1145/3543712.3543747 · Cited by 11
“According to Bengio [5], experiments showed that humans and animals learn better when the examples are presented in a suitable order.”
BackgroundInfluentialAutonomous Skill Learning of Water Polo Ball Heading for a Robotic Fish: Curriculum and Verification
Zhang, T., Wang, R., Wang, S., Wang, Y., Cheng, L., Zheng, G., & Tan, M. H. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3189095 · Cited by 11
Predicting Difficulty and Discrimination of Natural Language Questions
Byrd, M., & Srivastava, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.15 · Cited by 13
CurML: A Curriculum Machine Learning Library
Zhou, Y., Chen, H., Pan, Z., Yan, C., Lin, F., Wang, X., & Zhu, W. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548549 · Cited by 10
“For instance, Baby Step [31] and Lambda Step [13] add data to the training set from easy to hard in a predefined order.”
BackgroundInfluentialPaced-curriculum distillation with prediction and label uncertainty for image segmentation
Islam, M., Seenivasan, L., Sharan, S. P., Viekash, V. K., Gupta, B., Glocker, B., & Ren, H. (2023) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-023-02847-9 · Cited by 11
“Inspired by the cognitive process of humans and animals, numerous curriculum learning (CL) [1, 2] and self-paced learning (SPL) [3, 4] techniques are introduced in machine learning.”
BackgroundVæhrens, L., Álvarez, D. D., Berger, U., & Bøgh, S. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00201 · Cited by 13
“Bengio et al. [25] formulated curriculum learning as a particular case of transfer learning in which the training dataset is organized by sampling easier examples first rather than sampling from a uniform distribution.”
BackgroundOr, K., Wu, K., Nakano, K., Ikeda, M., Ando, M., Kuniyoshi, Y., & Niiyama, R. (2023) · Frontiers in Robotics and AI · doi:10.3389/frobt.2023.1066518 · Cited by 11
Dynamic Activation of Clients and Parameters for Federated Learning over Heterogeneous Graphs
Gu, Z., Zhang, K., Bai, G., Chen, L., Zhao, L., & Yang, C. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00126 · Cited by 22
“For example, inspired by cognitive science, curriculum learning [35] points out the pivotal role of training sample selection in machine learning tasks, which is now applied in graphs as well.”
BackgroundPMT-IQA: Progressive Multi-task Learning for Blind Image Quality Assessment
Pan, Q., Guo, N., Qingge, L., Zhang, J., & Yang, P. H. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-7025-4_13 · Cited by 13
Khanal, R., Choi, Y., & Lee, J. (2024) · Sensors · doi:10.3390/s24102977 · Cited by 17
Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?
Wang, F., Ding, L., Rao, J., Liu, Y., Shen, L., & Ding, C. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3690640 · Cited by 40
“, curriculum learning [78] and progressive learning [79], can be employed for the VLP, where the training starts with simple patterns and gradually goes into”
Transfer Incremental Learning Using Data Augmentation
Hacene, G. B., Gripon, V., Farrugia, N., Arzel, M., & Jézéquel, M. (2018) · Applied Sciences · doi:10.3390/app8122512 · Cited by 15
“This process is, most often, nondestructive, and results in what is often referred to as “curriculum learning” in the literature [1].”
BackgroundYang, Z., & Sudharshan, D. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.11.038 · Cited by 13
Gao, J., Du, J., & Chen, E. (2018) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2018.2886739 · Cited by 10
“based on the idea of curriculum learning [34] is used for speech enhancement [35], where each hidden layer of the DNN is designed to learn an intermediate target with an increased SNR.”
Adaptive Ensembling: Unsupervised Domain Adaptation for Political Document Analysis
Desai, S., Sinno, B., Rosenfeld, A., & Li, J. J. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1478 · Cited by 10
“To ease the difficulty of domain adaptation, we exploit the temporal information in our datasets to introduce a curriculum (Bengio et al., 2009).”
MethodMSPL: Multimodal Self-Paced Learning for Multi-Omics Feature Selection and Data Integration
Yang, Z., Xia, L.-Y., Zhang, H., & Liang, Y. Q. (2019) · IEEE Access · doi:10.1109/access.2019.2955958 · Cited by 11
“The fundamental definition of Curriculum Learning (CL) was first proposed by [31].”
BackgroundCognitive Robotics: Making Robots Sense, Understand, and Interact
Lange, D. (2019) · Computer · doi:10.1109/mc.2019.2942579 · Cited by 11
A new self-paced method for multiple instance boosting learning
Xiao, Y., Yang, X., & Liu, B. (2019) · Information Sciences · doi:10.1016/j.ins.2019.12.015 · Cited by 10
Modelling and Learning Dynamics for Robotic Food-Cutting
Mitsioni, I., Karayiannidis, Y., & Kragić, D. (2021) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case49439.2021.9551558 · Cited by 14
“A central part of the suggested training approach is curriculum training [25] which we combine with learning rate decay to avoid prediction error accumulation and facilitate training.”
BackgroundTraining Curricula for Open Domain Answer Re-Ranking
MacAvaney, S., Nardini, F. M., Perego, R., Tonellotto, N., Goharian, N., & Frieder, O. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401094 · Cited by 24
“Curriculum Learning (CL) can be considered a particular case of Continuation Methods , generally used when the target objective function is non-convex and its direct optimization may lead the training to converge to a poor local minimum [1, 4].”
BackgroundMethodLEAF: Latent Exploration Along the Frontier
Bharadhwaj, H., Garg, A., & Shkurti, F. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560922 · Cited by 18
“s during training is vital to LEAF. Hence, we leverage curriculum to ensure that in-distribution nearby goals are sampled before goals further away. This naturally relates to the idea of a curriculum [50], and can be ensured by gradually increasing the maximum horizon Hof the episodes. So, we start with a horizon leng…”
BackgroundIterative Label Improvement: Robust Training by Confidence Based Filtering and Dataset Partitioning
Haase-Schutz, C., Stal, R., Hertlein, H., & Sick, B. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9411918 · Cited by 15
“Other approaches dealing with label noise in the training material for deep neural networks are based on finding an optimal curriculum, to present the data, as first introduced in [17].”
BackgroundTargeted Data-driven Regularization for Out-of-Distribution Generalization
Kamani, M. M., Farhang, S., Mahdavi, M., & Wang, J. Z. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403131 · Cited by 13
“Mentornet classifies a dataset with noisy labels using the curriculum learning approach [8].”
BackgroundMethodCurriculum Learning for Wide Multimedia-Based Transformer with Graph Target Detection
Chen, W., Hong, F., Huang, C., Zhang, S., Wang, R., Xie, R., Xia, F., Lin, L., Zhang, Y., & Wang, Y. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3416275 · Cited by 10
“Bengio et al. proposed a new learning paradigm called curriculum learning (CL), in which a model is learned by gradually including from easy to complex samples during training process in order to increase the entropy of training samples[2].”
BackgroundCarpe Diem, Seize the Samples Uncertain "at the Moment" for Adaptive Batch Selection
Song, H., Kim, M., Kim, S., & Lee, J.-G. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3411898 · Cited by 20
“[1] first took easy samples and then gradually increased the difficulty of samples using heuristic rules.”
BackgroundSequence generative adversarial nets with a conditional discriminator
Yan, Y., Shen, G., Zhang, S., Huang, T., Deng, Z., & Yun, U. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.10.108 · Cited by 12
Leveraging Acoustic Images for Effective Self-supervised Audio Representation Learning
Sanguineti, V., Morerio, P., Pozzetti, N., Greco, D., Cristani, M., & Murino, V. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58542-6_8 · Cited by 12
“In particular, we exploit a curriculum learning scheme [5]: in the first epochs, we use all the triplets that contribute to the training (i.e. with a loss greater than zero), and later on only the hardest triplets: for each anchor, we select the hardest negative as the one with smallest distance,…”
MethodCategorical Perception: A Groundwork for Deep Learning
Bonnasse‐Gahot, L., & Nadal, J. (2021) · Neural Computation · doi:10.1162/neco_a_01454 · Cited by 18
“On the opposite side, taking inspiration from curriculum learning (Bengio et al., 2009), Morerio et al. (2017) propose the exact contrary: to start learning with low noise, and then increase the difficulty by raising the amount of noise.”
BackgroundFully Convolutional Geometric Features for Category-level Object Alignment
Feng, Q., & Atanasov, N. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341550 · Cited by 12
“We use the curriculum learning [35] to verify our cross-instance matching data augmentation idea so that the training procedure is more stable.”
MethodAutomated curriculum learning for embodied agents a neuroevolutionary approach
Milano, N., & Nolfi, S. (2021) · Scientific Reports · doi:10.1038/s41598-021-88464-5 · Cited by 14
Annotation Curricula to Implicitly Train Non-Expert Annotators
Lee, J.-U., Klie, J.-C., & Gurevych, I. (2022) · Computational Linguistics · doi:10.1162/coli_a_00436 · Cited by 22
“The learning curriculum is a fundamental concept in educational research that proposes to order exercises to match a learner’s proficiency (Vygotsky 1978; Krashen 1982) and has even motivated training strategies for machine learning models (Bengio et al. 2009).”
BackgroundPACE: Learning Effective Task Decomposition for Human-in-the-loop Healthcare Delivery
Zheng, K., Chen, G., Herschel, M., Ngiam, K. Y., Ooi, B. C., & Gao, J. (2021) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3448016.3457281 · Cited by 22
“Curriculum learning [7] can be viewed as a special form of transfer learning in that the chosen initial tasks guide the model to perform better in the final analytics [46].”
BackgroundP-DIFF+: Improving learning classifier with noisy labels by Noisy Negative Learning loss
Zhao, Q., Hu, W., Huang, Y., & Zhang, F. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.07.024 · Cited by 12
Adaptive Label Smoothing To Regularize Large-Scale Graph Training
Zhou, K., Choi, S.-H., Liu, Z., Liu, N., Yang, F., Chen, R., Li, L., & Hu, X. (2023) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611977653.ch7 · Cited by 15
“Motivated from the batch pacing in curriculum learning [3, 24], we propose a smooth pacing function to gradually schedule the appropriate smoothing strength α t at the t -th epoch.”
MethodAutomatic Curriculum Design for Object Transportation Based on Deep Reinforcement Learning
Eoh, G., & Park, T.-H. (2021) · IEEE Access · doi:10.1109/access.2021.3118109 · Cited by 10
“Curriculum learning is a step-wise training method for high performance and fast learning completion [53].”
MethodCrowder, D. C., Abreu, J., & Kirsch, R. F. (2021) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2021.3135471 · Cited by 11
“1) Curriculum Learning: Curriculum learning (Figure 1d) [25], [26] refers to initially teaching a controller an easy task and then teaching it progressively harder tasks.”
BackgroundYang, M., Xie, Z., Wang, Z., Yuan, Y., & Zhang, J. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3188326 · Cited by 14
Sun, L., Yan, J., & Qin, W. (2022) · Computer Animation and Virtual Worlds · doi:10.1002/cav.2113 · Cited by 10
“Curriculum learning 29 is a training strategy that trains the machine learning model starting with a small set of simple examples and finally with the whole training dataset.”
BackgroundLearning to Coordinate for a Worker-Station Multi-Robot System in Planar Coverage Tasks
Tang, J., Gao, Y., & Lam, T. L. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3214446 · Cited by 13
“stage curriculum learning [29] evolving from single worker into multiple workers, which guides workers and station for better policies during training.”
BackgroundShort text classification by detecting information path
Zhang, S., Jin, X., Shen, D., et al. (2013) · doi:10.1145/2505515.2505638 · Cited by 10
“…[8, 19] that improves the learning process by exploiting the unlabeled data in addition to the labeled training data, (2) transfer learning [9] that applies the knowledge learned from one domain to another domain, (3) curriculum learning [1] that learns the knowledge from easy to difficult gradually.”
BackgroundImproving discrimination ability of convolutional neural networks by hybrid learning
Kim, I., Choi, C., & Lee, S. (2015) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-015-0256-9 · Cited by 10
“Second, according to the theory of the curriculum learning, it is desirable to learn easy tasks first, and then, challenging tasks [23].”
BackgroundOne-Pass Logistic Regression for Label-Drift and Large-Scale Classification on Distributed Systems
Nguyen, V., Nguyen, T. D., Le, T., Venkatesh, S., & Phung, D. (2016) · Proceedings · doi:10.1109/icdm.2016.0145 · Cited by 10
“In contrast, shuffling data is an important factor to prevent stochastic gradient descent (SGD) from the bias optimization problem [12].”
BackgroundKellenberger, B., Marcos, D., & Tuia, D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-10997-4_40 · Cited by 10
“This is inspired by Curriculum Learning [1] and makes the CNN learn initial representations of both animals and background.”
MethodMulti-Robot Collision Avoidance with Map-based Deep Reinforcement Learning
Yao, S., Chen, G., Pan, L., Ma, J., Ji, J., & Chen, X. (2020) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai50040.2020.00088 · Cited by 12
“Curriculum learning [18] has recently been shown as an efficient strategy that can help find better local minima and accelerate the training process.”
BackgroundMethodConversations Powered by Cross-Lingual Knowledge
Sun, W., Meng, C., Meng, Q., Ren, Z., Ren, P., Chen, Z., & de Rijke, M. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462883 · Cited by 15
“…, we distillate the knowledge selection and knowledge expression ability from an auxiliary language to the target language; and in the curriculum learning part, we incorporate curriculum learning [3] into the distillation process to handle the noise caused by automatic parallel dialogue mining.”
BackgroundMethodSanchez, H. C., Hernández, N., Parra, I., Nebot, E., & Llorca, D. F. (2021) · IEEE Access · doi:10.1109/access.2021.3104340 · Cited by 11
Deep Q-Networks for Accelerating the Training of Deep Neural NetworksPreprint
Fu, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.01467 · Cited by 10
“In this paper, we consider the information associate with labels as the selection criteria, which is more close to curriculum learning [2, 8].”
MethodFast Adaptation in Generative Models with Generative Matching NetworksPreprint
Bartunov, S., & Vetrov, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.02192 · Cited by 24
“Such a training strategy is rooted in curriculum learning (Bengio et al., 2009) and meta-learning (Thrun, 1998; Vilalta & Drissi, 2002; Hochreiter et al., 2001).”
BackgroundMethodImproved Training with Curriculum GANsPreprint
Sharma, R., Barratt, S. T., Ermon, S., & Pande, V. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.09295 · Cited by 16
“The training setup of GANs bears little resemblance to the supervised curriculum learning studied in (Bengio et al., 2009), which further motivated our work to establish empirically the efficacy of a curriculum strategy when using GANs.”
BackgroundMethodInfluentialImproved Stereo Matching with Constant Highway Networks and Reflective Confidence LearningPreprint
Shaked, A., & Wolf, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.00165 · Cited by 10
“We believe that the reflective loss can be extended to other problems and applications such as the gradual learning schemes self-paced [19] and curriculum [1] learning.”
BackgroundAn Easy to Use Repository for Comparing and Improving Machine Learning Algorithm UsagePreprint
Smith, M. R., White, A. D., Giraud-Carrier, C. G., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.7292 · Cited by 10
“e effectiveness of the ensembling technique [13, 7, 1]. Recently, curriculum learning incrementally augments the training set such that “easier” instances are presented to the learning algorithm first [4]. Thus, creating a need to understand and identify the easier instances. Smith et al. used instance-level prediction…”
BackgroundImproved Mean Absolute Error for Learning Meaningful Patterns from Abnormal Training DataPreprint
Wang, X., Kodirov, E., Yang, H., et al. (2019) · arXiv (Cornell University) · Cited by 10
Progressive Learning and Disentanglement of Hierarchical RepresentationsPreprint
Li, Z., Murkute, J. V., Gyawali, P., & Wang, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10549 · Cited by 40
Speech Enhancement based on Denoising Autoencoder with Multi-branched EncodersPreprint
Yu, C., Zezario, R. E., Wang, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01538 · Cited by 10
Generalisation Guarantees for Continual Learning with Orthogonal Gradient DescentPreprint
Bennani, M., Doan, T., & Sugiyama, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11942 · Cited by 86
“Implications for Curriculum Learning We also observe that the upper bound depends on the task ordering, which may provide a theoretical explanation on the importance of learning with a curriculum (Bengio et al., 2009).”
BackgroundRevisiting Parameter Sharing in Multi-Agent Deep Reinforcement LearningPreprint
Terry, J. K., Grammel, N., Son, S., Black, B., & Agrawal, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.13625 · Cited by 197
Improving Medical Image Classification with Label Noise Using Dual-uncertainty EstimationPreprint
Ju, L., Wang, X., Wang, L., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.00528 · Cited by 9
“2) Curriculum training: We feed the selected clean samples ˆ X to the model at an early stage and then we have the network to begin learning all samples with a normalized weights in a curriculum-driven way [8].”
MethodSelf-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labelsPreprint
Peng, J., Wang, P., Desrosiers, C., & Pedersoli, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.13741 · Cited by 79
“Self-paced learning A sub-category of Curriculum Learning (CL) [1], Self-paced Learning (SPL) is inspired by the learning process of humans that gradually incorporates easy to hard samples in training [25].”
BackgroundWang, C., Karimzadeh, M., Ravindra, N. G., Bounds, L. R., Alerasool, N., Huang, A., Ma, S., Gulbranson, D. R., Cui, H., Lee, Y., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.18.712807 · Cited by 29
Deep reinforcement learning for community architectural layout generation
Sheng, T., Xiong, Y., Wang, H., Zhang, Y., Wang, S., & Zhang, W. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02291-4 · Cited by 15
“To help the convergence of multi-objective RL, we apply a curriculum learning strategy [16].”
Progressive Multistage Learning for Discriminative Tracking
Li, W., Li, X., Bourahla, O. E. F., Huang, F., Wu, F. Q., Liu, W., & Liu, H. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2985398 · Cited by 10
“Curriculum learning is a learning paradigm inspired by the intrinsic learning principle of humans/animals [47].”
BackgroundGenerating Texture for 3D Human Avatar from a Single Image using Sampling and Refinement Networks
Cha, S., Seo, K., Ashtari, A., & Noh, J. (2023) · Computer Graphics Forum · doi:10.1111/cgf.14769 · Cited by 13
“[7], the strategy has been adapted to various tasks, such as language modeling [12], object detection [41], and person re-identification [29].”
MethodCross-domain video action recognition via adaptive gradual learning
Liu, D., Bao, Z., Mi, J., Gan, Y., Ye, M., & Zhang, J. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126622 · Cited by 14
MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering
Pal, V., Yates, A., Kanoulas, E., & de Rijke, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.348 · Cited by 67
“We follow a Curriculum Learning approach (Bengio et al., 2009) by sequentially increasing the complexity of tasks to train MultiTabQA.”
Method
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.