Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 3,901–4,000 of 5,240
CrimeAlarm: Towards Intensive Intent Dynamics in Fine-Grained Crime Prediction
Hu, K., Li, L., Xie, Q., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5575-2_7 · Cited by 1
Efficient Subgraph Matching Framework for Fast Subcircuit Identification
Li, B., Wang, S., Chen, T., Sun, Q. L., & Zhuo, C. (2024) · ACM/IEEE Workshop on Machine Learning for CAD (MLCAD) · doi:10.1145/3670474.3685958 · Cited by 9
“In view of the significant variation in subgraph sizes across different hops, we introduce curriculum learning [30] to improve the convergence speed of the model.”
BackgroundA lightweight self-ensemble feedback recurrent network for fast MRI reconstruction
Li, J., Yang, H., Lui, L. M., et al. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02330-0 · Cited by 1
“Meanwhile, we also introduce the curriculum learning strategy [47] during training to further improve the performance of the model.”
Improved Communication and Collision-Avoidance in Dynamic Multi-Agent Path Finding
Xie, J., Zhang, Y., Ouyang, Q., Yang, H., Dong, F., Shi, D., & Jin, S. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651091 · Cited by 2
“Curriculum learning [31] is used to make the model learn incrementally, as shown in Fig.”
BackgroundMethodContinual Learning for Robust Gate Detection under Dynamic Lighting in Autonomous Drone Racing
Qiao, Z., Pham, H. X., Ramasamy, S., Jiang, X., Kayacan, E., & Sarabakha, A. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10649903 · Cited by 4
“Such find aligns with the idea of curriculum learning [30], that learning starting from easier to more complex tasks leads to better overall performance.”
BackgroundDeep Learning Based Lipreading Assistant
Somkuwar, S., Rathi, A., Todankar, S., & Yelalwar, R. (2024) · OPJU International Technology Conference on Smart Computing for Innovation and Advancement in Industry (OTCON) · doi:10.1109/otcon60325.2024.10687357 · Cited by 2
Jointly Recognizing Speech and Singing Voices Based on Multi-Task Audio Source Separation
Bai, Y., Li, C., Li, H., et al. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687477 · Cited by 1
“To address this, we propose a curriculum learning [29]-based two-stage training procedure.”
MethodDetecting chord tone alterations and suspensions
McLeod, A., & Rohrmeier, M. (2023) · Journal of New Music Research · doi:10.1080/09298215.2024.2412595 · Cited by 1
“…the CPM where we first train it on all data points, and then force it to focus on the more rare and difficult non-default chords, and showed that this training procedure, which we relate to both Curriculum Learning (Bengio et al., 2009) and Cognitive Load Theory (Sweller, 1994), improves accuracy.”
Learning-Based Two-Tiered Online Optimization of Region-Wide Datacenter Resource Allocation
Chen, C.-L., Zhou, H., Chen, J., Pedramfar, M., Lan, T., Zhu, Z., Zhou, C., Ruiz, P. M., Kumar, N., Dong, H., et al. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3484213 · Cited by 18
“The reward for reservation l at time slot t is given by [10] strategy is adopted.”
MethodZhuang, X., Mao, X. H., Jiang, Y.-H., Wu, H., Zhao, S., Cai, L., Liu, S., Chen, Y., Song, Y., Jia, C., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-9434-8_18 · Cited by 4
IOWA: An I/O-Aware Adaptive Sampling Framework for Deep Learning
Hu, S., Chen, W., Yin, Y., & He, S. (2024) · IEEE International Conference on Networking, Architecture and Storage (NAS) · doi:10.1109/nas63802.2024.10781364 · Cited by 1
Autonomous Mental Development at the Individual and Collective Levels: Concept and Challenges
Lippi, M., Mariani, S., Martinelli, M., & Zambonelli, F. (2024) · IEEE Access · doi:10.1109/access.2024.3522362 · Cited by 2
“Curriculum-based approaches to machine learning go somewhat in the direction of gradually developing the capability to act in complex scenarios [12].”
CurricularVPR: Curricular Contrastive Loss for Visual Place Recognition
Zhang, D., Chen, N., Wu, M., et al. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802279 · Cited by 1
“Curriculum learning [9] is a training strategy that sequences the training based on the difficulty level of samples.”
Lin, Z., Liu, G., Huang, X., et al. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3523429 · Cited by 1
“Curriculum learning (CL) [41] is utilized to train the model, where the initial parameters for the model with degree n are initialized from the parameters of the model with degree n − 1.”
Ordered sample weighting: Improving neural network training for complex image classification
Wang, Y., & Wu, W. (2025) · Discrete and Continuous Dynamical Systems - S · doi:10.3934/dcdss.2024210 · Cited by 1
Xie, W., Wang, X., Michalski, R., & Jia, T. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126482 · Cited by 1
CodeContrast: A Contrastive Learning Approach for Generating Coherent Programming Exercises
Torres, N. (2025) · Education Sciences · doi:10.3390/educsci15010080 · Cited by 3
Multi-Agent Multi-Target Tracking Using Curriculum Learning for Reinforcement Learning
Vincent, C. R., Henshall, A., Şenses, B., Ricard, M. J., & Karaman, S. (2025) · AIAA SciTech Forum · doi:10.2514/6.2025-1733 · Cited by 1
Zhang, Y., Yamakata, Y., & Tajima, K. (2025) · Journal of Information Processing · doi:10.2197/ipsjjip.33.104 · Cited by 1
Loss-Aware Curriculum Learning for Chinese Grammatical Error Correction
Zhang, D., Li, Y., Bai, L., Zhang, H., Li, Y., Lin, H., Zheng, H.-T., Su, X., & Shan, Z. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889130 · Cited by 1
“The core idea of Curriculum Learning (CL) is to facilitate the model’s training process by learning from simple samples to complex ones, which has demonstrated its efficacy in improving the performance of natural machine translation systems [12]–[16].”
Curriculum Learning aided Audio-Visual Speech Recognition with Arbitrary Speaker Number
Lin, Y., Jin, T., Cheng, X., Zhao, Z., & Wu, F. Q. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887654 · Cited by 1
“Curriculum learning is a type of learning strategy, in which learning starts with mini-batches sampled only from easy data, and increases the difficulty gradually [16].”
Ögretmen, L., Chen, M., Pitschi, P., & Lohmann, B. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919549 · Cited by 8
“[ 23], [24] III.”
Quadruped Robot Mastering Obstacle Avoidance: Integrating Perception and Decision-making
Zhao, K., Zhang, J., Liu, W., Lai, R., Guo, B., & He, L. (2024) · IEEE Smart World Congress (SWC) · doi:10.1109/swc62898.2024.00225 · Cited by 1
“Curriculum Learning [31] is an important learning strategy in the field of machine learning, characterized by an incremental progression from simpler instances to progressively more complex challenges.”
ComFace: Facial Representation Learning with Synthetic Data for Comparing Faces
Akamatsu, Y., Umematsu, T., Imaoka, H., Gomi, S., & Tsurushima, H. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00514 · Cited by 4
“Curriculum learning [5] mimics the human learning behavior of starting with simple tasks and gradually learning more complex concepts.”
Self-Recovery of Quadrupedal Robot Using Deep Reinforcement Learning
Kılıç, Y. E., Akar, Y. E., Temeltaş, H., et al. (2024) · International Conference on Robotics, Automation and Artificial Intelligence (RAAI) · doi:10.1109/raai64504.2024.10949561 · Cited by 1
Properties of neural networks identifying strongly lensed gravitational waves in time domain
Offermans, A., & Li, T. G. F. (2025) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2024.0169 · Cited by 1
Dual Agent Learning Based Aerial Trajectory Tracking
Garg, S., Masnavi, H., Fidan, B., & Janabi‐Sharifi, F. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3560841 · Cited by 3
“Curriculum learning is employed to train both policies effectively, starting with simpler tasks and progressingtocomplexones[35].”
Deep Reinforcement Learning in Labor Market Simulations
Chen, R., & Zhang, Z. (2025) · IEEE Conference on Computational Intelligence for Financial Engineering & Economics · doi:10.1109/cifer64978.2025.10975741 · Cited by 2
“As in most MARL models in economics, we implement curriculum learning to accelerate training [29], [30].”
Zhang, D., Bao, W., Ji, W., Zhang, X., Zhou, J., & Zhang, Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130430 · Cited by 2
Simultaneous Color Registration and Depth Completion of Point Clouds with Curriculum Learning
Martinez, J. C., Montes, A. M., Moreno, C. A. M., et al. (2025) · Sensors · doi:10.3390/s25113328 · Cited by 1
“We adopt a curriculum learning strategy [7,8] to train the network in stages of increasing difficulty.”
A new class correlation-based dynamic sample weighting method for medical image classification
Yi, G., Ling, M., Liu, X., Hai, Z., Li, Y., Han, M., Yang, C., Niu, L., Song, Y., & Liu, Q. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06690-0 · Cited by 2
Lee, C., Lee, H., & Whang, M. (2025) · Sensors · doi:10.3390/s25133995 · Cited by 4
Exploring Market Designs for Enhanced Flexibility Procurement with Deep Reinforcement Learning
Zobernig, V., Fanta, S., Strömer, S., Hemm, R., Stiasny, J., Cremer, J. L., & de Vries, L. J. (2025) · International Conference on the European Energy Market (EEM) · doi:10.1109/eem64765.2025.11050256 · Cited by 1
“To address policy training with long-term state dependencies, curriculum learning [27] provides a structured framework by incrementally training sub-policies.”
Yanyan, L. (2025) · International Journal of High Speed Electronics and Systems · doi:10.1142/s0129156425407430 · Cited by 1
Ma, W., Sun, C., Zhu, Y., Wang, Z., Zhao, X., Jing, M., Yu, J., & Tang, F. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3729986 · Cited by 4
Chen, Y., Wang, J., & Zhao, W. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-9881-3_29 · Cited by 1
Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotion
Bohlinger, N., Kinzel, J., Palenicek, D., Antczak, Ł., & Peters, J. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246447 · Cited by 10
“It should be noted that we were able to help the JTP approach learn by applying a curriculum [37] strategy on the sampled target velocities based on the tracking error, but we omit this strategy, as noisy estimates of the tracking error in real-world experiments make it unreliable.”
An approach for measuring semantic similarity between words using SVM and LS-SVM
Lavanya, S. R., & Arya, S. S. (2012) · International Conference on Computer Communication and Informatics (ICCCI) · doi:10.1109/iccci.2012.6158835 · Cited by 1
On Layer-Wise Representations in Deep Neural Networks
der Fakultät, V., & der Ingenieurwissenschaften, D. (2013) · Cited by 1
Self-paced Learning for Imbalanced Data
Zięba, M., Tomczak, J. M., & Świątek, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-662-49381-6_54 · Cited by 1
Face Recognition via Heuristic Deep Active Learning
Li, Y., Wang, K., Nie, L., & Wang, Q. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-69923-3_11 · Cited by 2
“The recently proposed self-paced learning (SPL) algo-rithm [1, 2] demonstrates the important role of high-certainty samples.”
BackgroundImproved classification model for peptide identification based on self-paced learning
Wang, Y., Liang, X., Xia, Z., et al. (2017) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2017.8217659 · Cited by 1
“The SPL is motivated by Curriculum learning and has been applied on training of non-convex objectives [11].”
BackgroundLeordeanu, M., & Sukthankar, R. (2017) · Springer series in cognitive and neural systems · doi:10.1007/978-3-319-29674-6_26 · Cited by 1
Mixture Self-Paced Learning for Multi-view K-Means Clustering
Yu, H. Z., Lian, Y., Xu, X., & Zhao, X. (2017) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/dasc-picom-datacom-cyberscitec.2017.193 · Cited by 1
“It is not only unrealistic in real application but also causing some problems due to the definition of ‘easy’ between human and computer may not be consistent [21].”
BackgroundCurriculum Learning for Depth Estimation with Deep Convolutional Neural Networks
Surendranath, A., & Jayagopi, D. B. (2018) · Mediterranean Conference on Pattern Recognition and Artificial Intelligence (MedPRAI) · doi:10.1145/3177148.3180085 · Cited by 3
“[1] has shown through experiments that curriculum learning not only improves the speed of convergence of the training process, but also provides an improved local minima.”
BackgroundMethodInfluentialHantrakul, L., Kondak, Z., & Weinberg, G. (2018) · International Conference on Movement and Computing (MOCO) · doi:10.1145/3212721.3212839 · Cited by 1
“Lastly, much like how a novice pianist learns a new musical instrument through etudes and exercises that become progressively harder, studies have shown how such an ordering of tasks is also beneficial to neural networks [13].”
BackgroundLow-Cost Breaking of a Unique Chinese Language CAPTCHA Using Curriculum Learning and Clustering
Stein, G., & Peng, Q. (2018) · IEEE International Conference on Electro Information Technology · doi:10.1109/eit.2018.8500113 · Cited by 1
Idiopathic Interstitial Pneumonias Medical Image Detection Using Deep Learning Techniques: A Survey
Sun, Z., & He, S. (2019) · ACM Southeast Regional Conference (ACMSE) · doi:10.1145/3299815.3314425 · Cited by 2
“In terms of human learning, it is believed that if the training set is not selected randomly, but selected in order according to certain rules, the training of neural network can be accelerated [25].”
BackgroundOnline selective training for faster neural network learning
Mourad, S., Vikalo, H., & Tewfik, A. H. (2019) · doi:10.1109/dsw.2019.8755604 · Cited by 1
“In [3], curriculum learning was proposed; it was shown there that using examples incurring lower loss in early training increases robustness to outliers and noisy data.”
BackgroundSelf-Paced Subspace Clustering
Liu, Y., Du, B., & Zhang, L. (2019) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2019.00068 · Cited by 1
“Motivated by human learning, curriculum learning [15] was proposed.”
BackgroundCross-Training Deep Neural Networks for Learning from Label Noise
Zhang, Z., Yang, J. K., Zhang, Z., et al. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803597 · Cited by 1
Spatio-temporal Active Learning for Visual Tracking
Liu, C., Zhu, P., & Hu, Q. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851981 · Cited by 1
“Curriculum learning(CL) [29] simulates the cognitive principles of human learning, it gradually selects samples from the data to learn from easy to complex based on prior knowledge.”
BackgroundAutonomous Navigation with Improved Hierarchical Neural Network Based on Deep Reinforcement Learning
Zhang, H., Qiu, T., Li, S., Zhu, C., Lan, X., & Chang, H. (2019) · Chinese Control Conference (CCC) · doi:10.23919/chicc.2019.8865201 · Cited by 1
A latent-label denoising method for relation extraction with self-directed confidence learning
Sun, T., Zhang, C., Ji, Y., et al. (2020) · Intelligent Data Analysis · doi:10.3233/ida-184414 · Cited by 1
“Hence we borrow some ideas from curriculum learning [25] and self-paced learning [11,12].”
Data Augmentation Under Scarce Condition for Neural Machine Translation
Luo, D., Shi, S., Su, R., & Huang, H. (2019) · IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS) · doi:10.1109/ccis48116.2019.9073698 · Cited by 1
End-to-End Optical Character Recognition Using Sythetic Dataset Generator for Noisy Conditions
Shopon, M., Diptu, N. A., & Mohammed, N. (2020) · Algorithms for intelligent systems · doi:10.1007/978-981-15-3607-6_41 · Cited by 1
A Perpetual Learning Algorithm That Incrementally Improves Performance With Deliberation
Qin, H., & Zhang, D. (2020) · IEEE Access · doi:10.1109/access.2020.3009718 · Cited by 1
“Y. Bengio et al . verified that the order of learning examples from easy to hard contributes to faster convergence and higher performance for prediction by experiments [37].”
BackgroundPlug-And-Pipeline: Efficient Regularization for Single-Step Adversarial Training
Vivek, B. S., Revanur, A., Venkat, N., & Babu, R. V. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00023 · Cited by 1
“One can perceive this as a curriculum [2], where the minimization of loss Lreg gradually occurs from i = 1 (easiest) to i = N (hardest).”
BackgroundMutual Constraint Learning for Weakly Supervised Object Detection
Liu, Y., Chen, W., Mahmud, S. M. H., Qu, H., Miao, K., Feng, W. Q., & Zhang, Z. (2019) · International Conference on Intelligent Systems and Knowledge Engineering (ISKE) · doi:10.1109/iske47853.2019.9170207 · Cited by 1
“For example, inspired by curriculum learning [1], [12] propose self-paced learning, which starts with easy samples, then consider hard ones in training.”
BackgroundLe, A. D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57058-3_26 · Cited by 1
“in 2009 [11].”
Training Confidence-Calibrated Classifier via Distributionally Robust Learning
Wu, H., & Wang, M. D. (2020) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac48688.2020.0-230 · Cited by 2
“We can also gradually decrease the value of α as training iterations grow, similar to the idea of curriculum training [18].”
BackgroundNon-convex Optimization Using Parameter Continuation Methods for Deep Neural Networks
Pathak, H. N., & Paffenroth, R. C. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-6759-9_12 · Cited by 1
Challenge Training to Simulate Inference in Machine Translation
Lu, W., Zhou, J., Zhou, L., Liu, G., & Zhang, Q. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206915 · Cited by 2
“Inspired by curriculum learning [6], previous works [3] [7] have proposed to transform the training process to relieve exposure bias.”
MethodA Noise Adaptive Model for Distantly Supervised Relation Extraction
Huang, X., Zhang, B., Ye, Y., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60450-9_41 · Cited by 1
Effective Use of Target-side Context for Neural Machine Translation
Mino, H., Ito, H., Goto, I., et al. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.396 · Cited by 1
“To train both target-side sentences with different features effectively, we propose a controlled sampling method inspired by curriculum learning approach (Bengio et al., 2009) and scheduled sampling (Bengio et al., 2015).”
MethodSasaya, T., Watanabe, T., Ono, T., et al. (2020) · International Conference on Condition Monitoring and Diagnosis (CMD) · doi:10.1109/cmd48350.2020.9287200 · Cited by 1
Learning to Generalise in Sparse Reward Navigation Environments
Jeewa, A., Pillay, A. W., & Jembere, E. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-66151-9_6 · Cited by 1
Towards a Task-Agnostic Model of Difficulty Estimation for Supervised Learning Tasks
Laverghetta, A., Mirzakhalov, J., & Licato, J. (2020) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing: Student Research Workshop (AACL-IJCNLP SRW) · doi:10.18653/v1/2020.aacl-srw.3 · Cited by 3
“A crucial question which recent AI work has not sufficiently addressed is how humans design curricula, which was stressed as essential to understand by early research in this area (Bengio et al., 2009; Taylor, 2009).”
BackgroundTowards Artifacts-Free Image Defogging
Graffieti, G., & Maltoni, D. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412289 · Cited by 1
“…real images and, at the same time, reducing the artifacts inserted during the defogging, we propose CurL-Defog , an approach inspired by curriculum learning [3], where the model is first guided towards a desirable parameters-space region via a pix2pix-like [13] training using synthetic paired image.”
MethodEfficient Conditional GAN Transfer with Knowledge Propagation across Classes
Shahbazi, M., Huang, Z., Paudel, D. P., Chhatkuli, A., & Gool, L. V. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01199 · Cited by 29
“It is well-established in [2, 13, 38, 19, 36, 11, 6] and many other works, that learning algorithms can greatly benefit from the shared knowledge between classes.”
BackgroundHow Important is Importance Sampling for Deep Budgeted Training?
Arazo, E., Ortego, D., Albert, P., O'Connor, N. E., & McGuinness, K. (2021) · British Machine Vision Conference (BMVC) · doi:10.5244/c.35.165 · Cited by 5
“This section reviews relevant works that aim to improve the computational efficiency of training DNNs. Curriculum learning (CL) aims to improve model performance by ordering the samples from easy to difficult [7, 13, 21, 56].”
Learning with Delayed Feedback
Pranavan, T., & Sim, T. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9411917 · Cited by 2
“In Curriculum Learning, the Agent learns a sequence of tasks that gets progressively harder [28], [29].”
BackgroundZhang, H., & Chan, W. K. (2021) · IEEE Transactions on Reliability · doi:10.1109/tr.2021.3086189 · Cited by 1
“Previous research has studied the effects of input data prioritization on reducing errors of DL models, such as using Curriculum Learning (CL) [3] and Self-paced Learning (SPL) [20].”
BackgroundMethodCan we have it all? On the Trade-off between Spatial and Adversarial Robustness of Neural Networks
Kamath, S., Deshpande, A., Subrahmanyam, K. V., & Balasubramanian, V. N. (2020) · Research Archive of Indian Institute of Technology Hyderabad (Indian Institute of Technology Hyderabad) · doi:10.48550/arxiv.2002.11318 · Cited by 16
“Curriculum learning is inherently inspired by how humans learn; first learning simple tasks and then followed by gradually difficult tasks [4].”
BackgroundDetect Noisy Label Based on Ensemble Learning
Chai, Y., Wu, C., & Zeng, J. (2021) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-030-70665-4_199 · Cited by 2
Curiosity Based RL on Robot Manufacturing Cell
Hameed, M. S. A., Khan, M. M., & Schwung, A. (2021) · IEEE International Conference on Industrial Technology (ICIT) · doi:10.1109/icit46573.2021.9453577 · Cited by 2
AdaReNet: Adaptive Reweighted Semi-supervised Active Learning to Accelerate Label Acquisition
Lourentzou, I., Gruhl, D., Alba, A., et al. (2021) · PErvasive Technologies Related to Assistive Environments Conference (PETRA) · doi:10.1145/3453892.3461321 · Cited by 1
“Inspired by Curriculum and Self-paced Learning [2, 29], recent work learns to re-weight examples [18, 23, 27, 36, 47].”
MethodTowards Robust Episodic Meta-Learning
Ermiş, B., Zappella, G., & Archambeau, C. (2021) · Cited by 4
REAL 2021 – Robot open-Ended Autonomous Learning: A Competition and Benchmark
Baldassarre, G., Montella, D., Santucci, V. G., & Cartoni, E. (2021) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/icdl49984.2021.9515624 · Cited by 1
Bougie, N., & Ichise, R. (2021) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-021-09527-5 · Cited by 3
A new strategy for curriculum learning using model distillation
Karaköse, K., & Bi̇lgi̇n, M. (2020) · Global Journal of Computer Sciences Theory and Research · doi:10.18844/gjcs.v10i2.5810 · Cited by 1
“When the data set examples are organised according to a curriculum, the model may find a better local minimum point [1].”
BackgroundMethodInfluentialAutomatic Recognition of Playful Physical Activity Opportunities of the Urban Environment
Saloheimo, T., Kaos, M., Fricker, P., & Hämäläinen, P. (2021) · International Academic Mindtrek Conference (Mindtrek) · doi:10.1145/3464327.3464369 · Cited by 2
“, a training curriculum where an initial learning task makes the final task easier [6].”
BackgroundHuxohl, T., & Kümmert, F. (2021) · Mathematics · doi:10.3390/math9192498 · Cited by 5
“For example, a kind of curriculum learning [24] can be used in which the self-training starts with objects where the model is certain in its predictions and then gradually continues to more uncertain, more difficult objects.”
BackgroundCompositional Data and Task Augmentation for Instruction Following
Dan, S., Han, X., & Roth, D. (2021) · doi:10.18653/v1/2021.findings-emnlp.178 · Cited by 1
“This approach is an instance of Curriculum Learning (Bengio et al., 2009) where the difficulty of an instance is related”
BackgroundOptimizing Univector Field Navigation Parameters Using CMA-ES
Baldão, T. P., Máximo, M. R. O. A., & Yoneyama, T. (2021) · Latin American Robotics Symposium, Brazilian Symposium on Robotics and Workshop on Robotics in Education (LARS/SBR/WRE) · doi:10.1109/lars/sbr/wre54079.2021.9605423 · Cited by 1
Provatas, N. (2021) · Very Large Data Bases · Cited by 2
GloCAL: Glocalized Curriculum-Aided Learning of Multiple Tasks with Application to Robotic Grasping
Kurkcu, A., Acar, C., Campolo, D., & Tee, K. P. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636492 · Cited by 2
Robust Classification of Histology Images Exploiting Adversarial Auto Encoders
Kurian, N. C., Singh, G., Hebbar, P., Kodate, S., Rane, S. U., & Sethi, A. (2021) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc46164.2021.9630477 · Cited by 1
“In curriculum learning the model is trained gradually using easier samples first, similar to how human are taught [7].”
MethodMulti-Scale, Super-Resolution Remote Imaging via Deep Conditional Normalizing Flows
Heintz, A. M., Peck, M. A., & Mackey, I. (2022) · AIAA SciTech Forum · doi:10.2514/6.2022-2499 · Cited by 1
Safe Curriculum Learning for Linear Systems with Parametric Unknowns in Primary Flight Control
Buysscher, D. D., Pollack, T., & Kampen, E.-J. V. (2022) · AIAA SciTech Forum · doi:10.2514/6.2022-0790 · Cited by 1
Training with growing sets: A comparative study
Yeğin, M. N., Kurttekin, Ö., Bahşi, S. K., & Amasyalı, M. F. (2022) · Expert Systems · doi:10.1111/exsy.12961 · Cited by 1
“Using this strategy, Bengio et al. (2009) suggested a method that composes the curriculum according to the human defined rules.”
BackgroundMethodAdaptive Hardness Indicator Softmax for Deep Face Recognition
Cai, M., Cheng, N., Cao, C., Yang, J., & Chen, Y. (2022) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001422560092 · Cited by 1
A First Approach to Image Transformation Sequence Retrieval
Mas-Candela, E., Ríos-Vila, A., & Calvo-Zaragoza, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-04881-4_26 · Cited by 1
A Human-machine Interface for Few-shot Rule Synthesis for Information Extraction
Vacareanu, R., Barbosa, G. C. G., Noriega-Atala, E., Hahn-Powell, G., Sharp, R., Valenzuela-Escárcega, M. A., & Surdeanu, M. (2022) · doi:10.18653/v1/2022.naacl-demo.8 · Cited by 1
“Further, we employ a curriculum learning approach (Bengio et al., 2009; Platanios et al., 2019), splitting the training data by sentence length and by pattern length.”
MethodCIAug: Equipping Interpolative Augmentation with Curriculum Learning
Sawhney, R., Soun, R., Pandit, S., Thakkar, M., Malaviya, S., & Pinter, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.127 · Cited by 1
“This notion ties in with the framework of curriculum learning (Krueger and Dayan, 2009), where training data is presented in a similarly staggered way, increasing model capabilities (Bengio et al., 2009).”
BackgroundPhaijit, O., Sammut, C., & Johal, W. (2022) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3527188.3561922 · Cited by 1
“It has been shown in research that finding a purposeful ordering of examples for the machine learner improves its learning process [3, 17, 20].”
BackgroundContextualizing Lung Nodule Malignancy Predictions with Easy vs. Hard Image Classification
Xu, E., Ramaraj, T., Tchoua, R., Furst, J. D., & Raicu, D. S. (2022) · International Conference on Transdisciplinary AI (TransAI) · doi:10.1109/transai54797.2022.00028 · Cited by 1
“Earlier strategies include creating discrete data sets that gradually increase in difficulty [2], [10] while other, more recent works continuously and dynamically increment the difficulty of the training set on each epoch [3], [11].”
BackgroundSelf-paced Learning for Pedestrian Trajectory Prediction
Wu, Y., Li, B., Zhang, R., et al. (2022) · International Conference on Advanced Robotics and Mechatronics (ICARM) · doi:10.1109/icarm54641.2022.9959497 · Cited by 1
“Curriculum Learning [7] is a new training strategy proposed by Bingo et al. that progressive learns samples by mimicking the human ’easy to hard’ learning process.”
BackgroundMethodVideo Captioning model using a curriculum learning approach
Varma, S., & Peter, J. D. (2022) · International Conference on Innovations in Science and Technology for Sustainable Development (ICISTSD) · doi:10.1109/icistsd55159.2022.10010501 · Cited by 1
“The concept of Curriculum Learning as proposed by Bengio et.al [26] refers to the training strategy using which a machine learning model could be learnt with a curriculum.”
BackgroundMethodMahmoud, S., Billing, E., Svensson, H., & Thill, S. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1098982 · Cited by 1
“Specifically, curriculum learning suggests that it is beneficial to structure the exposure to learning examples, often progressively proceeding from initially simple tasks to increasingly complex ones (Elman, 1993; Bengio et al., 2009; Krueger and Dayan, 2009; Hacohen and Weinshall, 2019).”
Background
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.