Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 3,501–3,600 of 5,240
Fine-grained Pseudo Labels for Scene Text Recognition
Li, X., Chen, X., Huang, Z., Xie, L., Chen, J., & Yang, M. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611791 · Cited by 3
“Curriculum learning was firstly introduced in [7].”
Liao, R., Zhang, Y., Yan, J., Cai, Y., & Tao, N. (2023) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2023.103787 · Cited by 2
Optimizing Predictive Analytics in 5G Networks Through Zero-Trust Operator-Customer Cooperation
Milani, M., Bega, D., Gramaglia, M., et al. (2023) · IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN) · doi:10.1109/nfv-sdn59219.2023.10329622 · Cited by 2
“This is similar to implementing a curriculum learning approach [16].”
MethodLeveraging Domain Adaptation and Data Augmentation to Improve Qur’anic IR in English and Arabic
Pavlova, V. (2023) · Arabic Natural Language Processing Conference (ArabicNLP) · doi:10.18653/v1/2023.arabicnlp-1.7 · Cited by 8
“In order to accommodate the limited size of the pre-training corpus, we schedule two-stage pre-training akin to phases of Curriculum learning (Bengio et al., 2009; Soviany et al., 2022).”
MethodInference Discrepancy Based Curriculum Learning for Neural Machine Translation
Zhou, L., Sasano, R., & Takeda, K. (2023) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2023edp7048 · Cited by 3
Integrating curriculum learning with meta-learning for general rhetoric identification
Wang, D., Li, Y., Wang, S., et al. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-023-02038-7 · Cited by 2
“On the other hand, inspired by human and animal learning, curriculum learning [10, 11] posits that neural networks are better trained when examples are not randomly presented but instead organized in a meaningful order that gradually shows more concepts and complexity, similar to the way humans…”
Unbiased, Effective, and Efficient Distillation from Heterogeneous Models for Recommender Systems
Kang, S., Kweon, W., Lee, D., Lian, J., Xie, X., & Yu, H. (2024) · ACM Transactions on Recommender Systems · doi:10.1145/3649443 · Cited by 5
“It trains the model by using easy samples first and progressively more difficult samples so that the model can leverage previously seen concepts to ease the acquisition of more difficult ones [ 3 , 16 ].”
InfluentialDynamic Data Sampler for Cross-Language Transfer Learning in Large Language Models
Li, Y., Feng, Y., Wen, Z., Zhao, Z., Shen, L., Hou, C., & Hou, X. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446640 · Cited by 7
“Moreover, inspired by curriculum learning[5], we propose a dynamic data sampler that progressively transition the model from unsupervised pre-training to supervised fine-tuning using training data with dynamic distribution.”
MethodAdaptive algorithms for shaping behavior
Tong, W. L., Iyer, A., Murthy, V. N., & Reddy, G. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013454 · Cited by 3
“In machine learning, the importance of shaping-inspired approaches for training agents was recognized early on [14–20].”
Localizing and Visualizing Relative Attributes
Xiao, F., & Lee, Y. J. (2017) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-50077-5_7 · Cited by 2
Direct Optimization of F-Measure for Retrieval-Based Personal Question Answering
Fakoor, R., Kainth, A., Shakeri, S., Winestock, C., Mohamed, A., & Sarikaya, R. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639562 · Cited by 2
“To resolve this problem, we use curriculum learning [26] under a multi-task learning (MTL) framework.”
MethodPerceptual Face Completion using a Local-Global Generative Adversarial Network
Ma, R., & Hu, H. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8545559 · Cited by 2
“Training is done by the curriculum learning method [4] and we use Adam [13] for optimization.”
MethodMaMiC: Macro and Micro Curriculum for Robotic Reinforcement Learning
Tomar, M., Sathuluri, A., & Ravindran, B. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.1609/aaai.v33i01.330110053 · Cited by 4
“Curriculum learning [3], [23] is a powerful concept that allows us to come up with such training strategies.”
Boosting learning to rank with user dynamics and continuation methods
Ferro, N., Lucchese, C., Maistro, M., & Perego, R. (2019) · Information Retrieval · doi:10.1007/s10791-019-09366-9 · Cited by 3
“Learning through a curriculum is an interesting idea, borrowed from cognitive science, according to which a complex learning process is designed as a multi-step training path (Bengio et al., 2009).”
BackgroundInfluentialCurriculum learning for distant supervision relation extraction
Liu, Q., Wang, P., Wang, J., et al. (2020) · Journal of Web Semantics · doi:10.1016/j.websem.2020.100559 · Cited by 2
Optimized CNN with Point-Wise Parametric Rectified Linear Unit for Spatial Image Steganalysis
Xue, Y., Peng, W., Wang, Y., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-43575-2_3 · Cited by 2
Unsupervised Visual Learning: From Pixels to Seeing
Leordeanu, M. (2020) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-030-42128-1_1 · Cited by 3
Using Concepts to Understand Intelligent Agents.
Meyer, O., Hesenius, M., & Gruhn, V. (2020) · AAAI Spring Symposium Combining Machine Learning with Knowledge Engineering · Cited by 2
Document-Level Neural TTS Using Curriculum Learning and Attention Masking
Hwang, S.-W., & Chang, J. (2021) · IEEE Access · doi:10.1109/access.2020.3049073 · Cited by 2
“Curriculum learning is a deep learning method first proposed in [14]; it starts with simple data and then the difficulty of learning increases with increasingly complex data.”
BackgroundMethodRobust Collaborative Learning with Noisy Labels
Sun, M., Xing, J., Chen, B., & Zhou, J. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00163 · Cited by 6
“Inspired by the fact that humans learn better when trained with a curriculum-like strategy, [21] first proposed curriculum learning.”
BackgroundCamargo, J. E., & Sáenz, R. (2021) · INTELIGENCIA ARTIFICIAL · doi:10.4114/intartif.vol24iss68pp1-20 · Cited by 2
“In this document we present the results of several experiments with curriculum learning [1] applied to a game AI learning process to measure its effects on the learning time, specifically we trained an agent using a reinforcement learning [19] algorithm to play a video game running on a game simulation platform, then…”
BackgroundMethodInfluentialHow State-Of-The-Art Models Can Deal With Long-Form Question Answering
Bui, M.-Q. V., Tran, V., Nguyen, H.-T., et al. (2020) · Pacific Asia Conference on Language, Information, and Computation · Cited by 2
An Empowerment-based Solution to Robotic Manipulation Tasks with Sparse Rewards
Dai, S., Xu, W., Hofmann, A., & Williams, B. C. (2021) · Robotics: Science and Systems · doi:10.15607/rss.2021.xvii.001 · Cited by 7
Friendly Training: Neural Networks Can Adapt Data To Make Learning Easier
Marullo, S., Tiezzi, M., Gori, M., & Melacci, S. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534165 · Cited by 4
“The idea of training neural networks with a learning methodology that “gradually” changes the learning environment (which proves to be appropriate not only in humans but also in animals [16]) traces back to almost three decades ago [17] and has been known for long as Curriculum Learning (CL) [8].”
BackgroundCurriculum Learning Revisited: Incremental Batch Learning with Instance Typicality Ranking
Krysińska, I., Morzy, M., & Kajdanowicz, T. J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86380-7_23 · Cited by 2
CurL-AutoML: Curriculum Learning-based AutoML
Silva, L. N. D. S., Silva, L. C., Zagatti, F. R., Sette, B. S., Caseli, H. D. M., Lucrédio, D., & Silva, D. F. (2021) · International Conference on Machine Learning and Applications · doi:10.1109/icmla52953.2021.00127 · Cited by 2
Leveraging Curriculum Reinforcement Learning for Rocket Powered Landing Guidance and Control
Yuan, H., Zhao, Y., Mou, Y., & Wang, X. (2021) · China Automation Congress (CAC) · doi:10.1109/cac53003.2021.9728176 · Cited by 2
Parallel Classification of Cells in Thinprep Cytology Test Image for Cervical Cancer Screening
Cao, M., Zhang, X., Fan, X., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17979-2_4 · Cited by 2
Factorial Hidden Markov Models for Learning Representations of Natural LanguagePreprint
Nepal, A., & Yates, A. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6168 · Cited by 2
“…similarity [9, 42, 40] as non-sparse features [38, 10, 35, 53]; 4) and recently, language models [3, 41] as representations [12, 52, 11, 2], some of which have already yielded state of the art performance on domain adaptation tasks [27, 29, 30, 49, 28] and information extraction tasks…”
BackgroundProbabilistic Models of Student Learning and Forgetting
Lindsey, R. (2014) · CU Scholar (University of Colorado Boulder) · Cited by 4
Faster Reinforcement Learning Using Active SimulatorsPreprint
Jain, V., & Tulabandhula, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07853 · Cited by 2
“In [4], the authors showed that learning using a curriculum has an effect on the rate of convergence of the prediction model.”
BackgroundTempered Adversarial NetworksPreprint
Sajjadi, M. S. M., Parascandolo, G., Mehrjou, A., & Schölkopf, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04374 · Cited by 21
“Bengio et al. (2009) coin the term curriculum learning where the idea is to present the samples during training in a specific order that improves the learning process.”
BackgroundOn the Intrinsic Dimensionality of Image RepresentationsPreprint
Gong, S., Boddeti, V. N., & Jain, A. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09672 · Cited by 2
“We adopt a curriculum learning [3] approach to overcome this challenge and progressively reduce the dimensionality of the mapping in multiple stages.”
BackgroundMethodDistribution Aware Active LearningPreprint
Mehrjou, A., Khodabandeh, M., & Mori, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08916 · Cited by 3
“Here we introduce a dynamical approach inspired by non-autonomous dynamical systems that changes the problem setting over time [21, 22, 23].”
MethodUnderstanding Self-Paced Learning under Concave Conjugacy TheoryPreprint
Liu, S., Ma, Z., & Meng, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08096 · Cited by 2
“Since being raised recently, self-paced learning (SPL)[1] and curriculum learning (CL)[2] have been grabbing attention in machine learning and arti-ficial intelligence.”
BackgroundMethodAdversarial Exploration Strategy for Self-Supervised Imitation LearningPreprint
Hong, Z.-W., Fu, T.-J., Shann, T.-Y., Chang, Y.-H., & Lee, C. (2018) · arXiv (Cornell University) · Cited by 2
Hu, G., Cui, B., & Yu, S. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.04237 · Cited by 2
“Some previous studies adopt heuristic rules to adjust the sampling probabilities of the train data, such as curriculum learning (Bengio et al. 2009), self-paced learning (Kumar, Packer, and Koller 2010), online batch selection (Loshchilov and Hutter 2015), etc. Fan et al .”
MethodUnderstanding and Controlling Memory in Recurrent Neural NetworksPreprint
Haviv, D., Rivkind, A., & Barak, O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07275 · Cited by 29
“We found that, in good accordance with existing literature (Bengio et al., 2009; Jozefowicz et al., 2015) results for the nominal test-set were fairly indifferent to the training protocol (Supplementary material).”
InfluentialProgressive Ensemble Networks for Zero-Shot RecognitionPreprint
Ye, M., & Guo, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07473 · Cited by 2
“In [4], progressive curriculum learning is used to train a model on "easy-to-hard" samples with a pre-defined scheme.”
MethodGradual Machine Learning for Aspect-level Sentiment AnalysisPreprint
Wang, Y., Chen, Q., Shen, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02502 · Cited by 2
“The other related machine learning paradigms include curriculum learning (CL) [10] and self-paced learning (SPL) [21].”
Transfer Incremental Learning using Data AugmentationPreprint
Hacene, G. B., Gripon, V., Farrugia, N., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02020 · Cited by 2
“This process is, most often, nondestructive, and results in what is often referred to as “curriculum learning” in the literature [1].”
BackgroundAdversarial Learning of Semantic Relevance in Text to Image SynthesisPreprint
Cha, M., Gwon, Y., & Kung, H. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.05083 · Cited by 2
“As a systematic way to provide negative examples of incremental semantic difficulty, we use curriculum learning (Bengio et al. 2009) by gradually increasing the semantic similarity between the input encoded text and negative image.”
BackgroundMethodInfluentialEnsemble Teaching for Hybrid Label PropagationPreprint
Gong, C., Tao, D., Chang, X., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03828 · Cited by 2
Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-CorrectionPreprint
Zhu, F., Chang, X., Zeng, R., & Tan, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.09205 · Cited by 2
“[3] propose a continuation method, namely curriculum learning, to solve complex sequences of tasks.”
HeMIS: Hetero-Modal Image SegmentationPreprint
Havaei, M., Guizard, N., Chapados, N., & Bengio, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.05194 · Cited by 10
“Inspired by previous works on curriculum learning [1]—where the model starts learning from easy scenarios before turning to more difficult ones—we used a pseudo-curriculum learning scheme where after few warmup epochs where all modalities are shown to the model, we start randomly dropping modalities,…”
MethodSelf-citationFixed-point optimization of deep neural networks with adaptive step size retrainingPreprint
Shin, S., Boo, Y., & Sung, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08171 · Cited by 2
“The curriculum learning is a training strategy to move the goal from an easy level to more complex one gradually [13].”
BackgroundCurriculum Learning for Deep Generative Models with ClusteringPreprint
Zhao, D., Zhu, J., Guo, Z., & Zhang, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.11594 · Cited by 4
“This idea of progressive learning is actually a general manner of cognition process Elman [1993], Oudeyer et al. [2007], which has been formally named curriculum learning in machine learning Bengio et al. [2009].”
BackgroundMethodProbabilistic Rollouts for Learning Curve Extrapolation Across Hyperparameter SettingsPreprint
Gargiani, M., Klein, A., Falkner, S., & Hutter, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.04522 · Cited by 19
“Regarding the experiments with the VRNN model, in order to speed up the training procedure, we also used a curriculum learning based technique (Bengio et al., 2019) and linearly increased the length of the input sequence during training, starting from a selected number of initial observed epochs (this hyperparameter,…”
MethodSelf-Paced Multi-Label Learning with DiversityPreprint
Seyedi, S. A., Ghodsi, S., Akhlaghian, F., Jalili, M., & Moradi, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.03497 · Cited by 7
“Curriculum and self-paced learning (SPL) are recently proposed regimes with the aim of learning from easier to more complex concepts Bengio et al. (2009); Kumar et al. (2010); Meng et al. (2017). These learning frameworks are inspired by human education system where the major difference arises in identifying the compl…”
BackgroundMethodInfluentialMIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech RecognitionPreprint
Chang, X., Zhang, W., Qian, Y., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.06522 · Cited by 2
“According to previous researches, starting from easier subtasks can lead the model to learn better, an approach called curriculum learning [30, 31].”
BackgroundA Deep Reinforcement Learning Architecture for Multi-stage Optimal ControlPreprint
Yang, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.10684 · Cited by 5
“The stacking structure and the backward training scheme resembles the reverse curriculum learning[8, 15, 29].”
MethodWang, X., Kodirov, E., Yang, H., & Robertson, N. M. (2019) · arXiv (Cornell University) · Cited by 3
Abuduweili, A., & Liu, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.01790 · Cited by 2
“proach uses stochastic gradient descent (SGD) to adapt the model online Bhasin et al. (2012). However, these methods may be sub-optimal in minimizing the local prediction errors. Another solution is to use the recursive least square parameter adaptation algorithm (RLS-PAA) Ljung and Priouret (2010), which has been app…”
MethodInfluentialCNN-LSTM models for Multi-Speaker Source Separation using Bayesian Hyper Parameter OptimizationPreprint
Zegers, J., & hamme, H. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.09254 · Cited by 3
“Curriculum learning was used [26], i.”
Learning from Synthetic AnimalsPreprint
Mu, J., Qiu, W., Hager, G. D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.08265 · Cited by 2
“To tackle the problem, we introduce a curriculum learning strategy [2, 13, 16] to progressively increase pseudo-labels and train models in iterations.”
MethodTransferring Optimality Across Data Distributions via Homotopy Methods
Gargiani, M., Zanelli, A., Dinh, Q. T., et al. (2020) · FreiDok plus (Universitätsbibliothek Freiburg) · Cited by 2
Learning Neural ActivationsPreprint
Minhas, F., & Asif, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.12187 · Cited by 3
“As a consequence, it may be advisable to have fixed initial linear activation functions in the final layers when initializing the neural network and gradually making them more and more non-linear similar to ideas underlying curriculum learning [14].”
BackgroundBuerkle, N. P., & Palmer, S. E. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2019.12.30.891382 · Cited by 2
“Surprisingly, evolved networks inherited this improved performance, highlighting the potential value of curriculum learning (Bengio et al., 2009) in an evolutionary context.”
BackgroundDesign of Capacity-Approaching Low-Density Parity-Check Codes using Recurrent Neural NetworksPreprint
Nisioti, E., & Thomos, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01249 · Cited by 5
“This idea was motivated by curriculum learning, a machine learning technique according to which an optimizer is presented with data of increasing difficulty in order to gradually manage to solve harder problems [31].”
MethodSelf-supervised visual feature learning with curriculumPreprint
Keshav, V., & Delattre, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.05634 · Cited by 4
“We discuss our approach to devise a better self-supervised feature learning strategy commonly applied to supervised training called curriculum learning [1].”
MethodScale-Invariant Multi-Oriented Text Detection in Wild Scene ImagesPreprint
Dasgupta, K., Das, S. K., & Bhattacharya, U. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06423 · Cited by 2
“Also, we use curriculum learning [24] by manually arranging images from simpler to difficult ones.”
MethodRepresentation Learning for Remote Sensing: An Unsupervised Sensor Fusion ApproachPreprint
Swope, A., Rudelis, X., & Story, K. T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.05094 · Cited by 24
“This is a form of curriculum learning (Bengio et al., 2009) where the contrastive task starts in an easier setting where more band are retained, and gets gradually harder throughout training; we observed faster convergence with this channel dropout schedule.”
BackgroundAdversarial Active Exploration for Inverse Dynamics Model LearningPreprint
Hong, Z.-W., Fu, T.-J., Shann, T.-Y., Chang, Y.-H., & Lee, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.10019 · Cited by 7
“This phenomenon indicates that adversarial active exploration naturally gives rise to a curriculum that improves the learning efficiency, which resembles curriculum learning [34].”
BackgroundLearning Latent Characteristics of Data and Models using Item Response Theory
Lalor, J. P. (2020) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.7275/reha-je40 · Cited by 4
Applying Cyclical Learning Rate to Neural Machine TranslationPreprint
Lee, C. M., Liu, J., & Peng, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.02401 · Cited by 3
“CLR draws its inspiration from curriculum learning (Bengio et al., 2009) and simulated annealing (Aarts and Korst, 2003).”
BackgroundReinforced Curriculum Learning on Pre-trained Neural Machine Translation ModelsPreprint
Zhao, M. M., Wu, H., Niu, D., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.05757 · Cited by 2
“Without surprise, curriculum learning is found most helpful in end-to-end neural network architectures (Bengio et al. 2009), since the performance that an artificial neural network can achieve critically depends on the quality of training data presented to it.”
InfluentialCLOPS: Continual Learning of Physiological SignalsPreprint
Kiyasseh, D., Zhu, T., & Clifton, D. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.09578 · Cited by 11
“These decisions are influence by curriculum learning [Bengio et al., 2009] and the intuition that training on similar tasks is less likely to lead to destructive interference.”
BackgroundMethodDifficulty Translation in Histopathology ImagesPreprint
Wei, J., Suriawinata, A. A., Liu, X., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12535 · Cited by 2
“Future research could explore difficulty translation in the context of curriculum learning [29] or defending against adversarial attacks [30].”
BackgroundAssessing the Bilingual Knowledge Learned by Neural Machine Translation ModelsPreprint
He, S., Wang, X., Shi, S., Lyu, M. R.-T., & Tu, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.13270 · Cited by 3
“These results also indicate that NMT models might by nature has the learning ability similar to the curriculum learning (Bengio et al., 2009; Kocmi and Bojar, 2017) without any explicit curriculum.”
ResultA Conceptual Framework for Lifelong LearningPreprint
Ling, C. X., & Bohn, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.09704 · Cited by 4
“Through leveraging curriculum learning [14] at both the level of task samples and perhaps individual tasks, the degree of forward transfer and few-shot learning capacity may be further improved.”
BackgroundCurriculum learning for multilevel budgeted combinatorial problemsPreprint
Nabli, A., & Carvalho, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.03151 · Cited by 8
“One of the strategies used to learn to solve complex problems is to break them down in a sequence of learning tasks that are increasingly harder, a concept known as Curriculum Learning [7].”
BackgroundInfluentialCurriculum Learning Strategies for IR: An Empirical Study on Conversation Response RankingPreprint
Penha, G., & Hauff, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.08555 · Cited by 16
“More recently, Bengio et al. [1] provided additional evidence that curriculum strategies can benefit neural network training with experimental results on different tasks such as shape recognition and language modelling.”
BackgroundInfluentialDecoupling Inherent Risk and Early Cancer Signs in Image-based Breast Cancer Risk ModelsPreprint
Liu, Y., Azizpour, H., Strand, F., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05791 · Cited by 2
“Networks are known to converge faster with easier examples [20, 21], and if it focuses too strongly on increasing confidence of the easy samples [22] learning on the harder long-term risk problem may be crippled.”
BackgroundFocus-and-Expand: Training Guidance Through Gradual Manipulation of Input FeaturesPreprint
Arar, M., Fish, N., Daniel, D., Tenetov, E., Shamir, A., & Bermano, A. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07723 · Cited by 2
“Curriculum Learning (CL) techniques impose structure on the training data by sorting the samples from simple to hard, and using this ordering to progressively train the learner [4].”
BackgroundImproving Monocular Depth Estimation by Leveraging Structural Awareness and Complementary DatasetsPreprint
Chen, T., An, S., Zhang, Y., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.11256 · Cited by 2
“When training on mixed datasets, curriculum learning [3] is needed to avoid the local minimum problem by training the model on easier datasets first.”
BackgroundMethodCurriculum Learning with a Progression Function
Bassich, A., Foglino, F., Leonetti, M., & Kudenko⋆, D. (2020) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.2008.00511 · Cited by 5
Follow the Object: Curriculum Learning for Manipulation Tasks with Imagined GoalsPreprint
Kilinc, O., & Montana, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.02066 · Cited by 4
“Curriculum learning is based upon the principle that mastering simple skills first can help master harder ones later [18].”
BackgroundAgent Environment Cycle GamesPreprint
Terry, J. K., Grammel, N., Black, B., Hari, A., Horsch, C., & Santos, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.13051 · Cited by 7
“Curriculum learning is a technique where progressively more complicated mechanics are added to an environment to make it easier to learn very complex behaviors [1].”
BackgroundNeural Machine Translation: A ReviewPreprint
Stahlberg, F. (2019) · Journal of Artificial Intelligence Research · doi:10.48550/arxiv.1912.02047 · Cited by 459
“An alternative to hard data filtering called curriculum learning [467] that controls the order of training samples has been applied to NMT by van der Wees et al.”
MethodBeyond Tabula-Rasa: a Modular Reinforcement Learning Approach for Physically Embedded 3D SokobanPreprint
Karkus, P., Mirza, M., Guez, A., Jaegle, A., Lillicrap, T., Buesing, L., Heess, N., & Weber, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01298 · Cited by 8
“Domain knowledge may appear in various forms, including reward shaping [17, 18], training curricula [19, 20], auxiliary tasks [21], state representation [22], algorithmic structure [23, 24, 12] and memory [25, 26].”
BackgroundAn Empowerment-based Solution to Robotic Manipulation Tasks with Sparse RewardsPreprint
Dai, S., Xu, W., Hofmann, A., & Williams, B. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07986 · Cited by 2
Adam with Bandit Sampling for Deep LearningPreprint
Liu, R., Wu, T., & Mozafari, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12986 · Cited by 17
“Curriculum learning [5, 17] is another optimization strategy that leverages a pre-trained teacher model to train the target model.”
BackgroundNegative-ResNet: Noisy Ambulatory Electrocardiogram Signal Classification SchemePreprint
Chen, Z., Lin, Z., Wang, P., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.10061 · Cited by 2
“Other approaches including dataset pruning [37]–[39], sample choosing [40]–[43], sample importance weighting [44]– [47] also show significant improvement on noisy data classification.”
BackgroundActive Screening for Recurrent Diseases: A Reinforcement Learning ApproachPreprint
Ou, H.-C., Chen, H., Jabbari, S., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.02766 · Cited by 2
Deep Adversarial Inconsistent Cognitive Sampling for Multi-view Progressive Subspace ClusteringPreprint
Sun, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.03783 · Cited by 2
“Self-paced learning derives from curriculum learning [35], a human-like learning principle where the easier instances are learned first, and then more difficult instances are gradually introduced to the learning process.”
BackgroundEmbodied Continual Learning Across Developmental Time Via Developmental Braitenberg Vehicles.Preprint
Alicea, B., Chakrabarty, R., Gopi, A., Lim, A., & Parent, J. (2021) · arXiv (Cornell University) · Cited by 4
Spirit Distillation: Precise Real-time Semantic Segmentation of Road Scenes with Insufficient DataPreprint
Wu, Z., Jiang, Y. Y., Cui, C., Yang, Z., Xue, X., & Qi, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.13733 · Cited by 6
“The idea of curriculum learning [32] suggests that prioritizing the learning of simple concepts when learning complex ones tend to be beneficial in improving the final prediction effect of machine learning models.”
BackgroundMethodAbstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution
Zhang, C., Jia, B., Zhu, S. C., & Zhu, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.14230 · Cited by 2
“Based on this observation, we train our PrAE learner in a 3-stage curriculum [2].”
MethodA Deep Learning Framework for Lifelong Machine LearningPreprint
Ling, C. X., & Bohn, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.00157 · Cited by 6
“Special cases of LML are multi-task learning (Caruana, 1997) and curriculum learning (Bengio et al., 2009).”
BackgroundDecentralized Swarm Collision Avoidance for Quadrotors via End-to-End Reinforcement Learning.
Ourari, R., Cui, K., & Koeppl, H. (2021) · TUbilio (Technical University of Darmstadt) · Cited by 2
MAFER: a Multi-resolution Approach to Facial Expression RecognitionPreprint
Massoli, F. V., Cafarelli, D., Gennaro, C., Amato, G. M., & Falchi, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.02481 · Cited by 8
“The first one is related to the use of the curriculum learning paradigm [4], i.”
BackgroundLearning to Aggregate and Personalize 3D Face from In-the-Wild Photo CollectionPreprint
Zhang, Z., Ge, Y., Chen, R., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07852 · Cited by 2
“This motivates us to perform a curriculum learning procedure [3, 17], i.”
MethodUnsupervised Path Representation Learning with Curriculum Negative SamplingPreprint
Yang, B., Guo, C., Hu, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09373 · Cited by 2
“For example, Deepcas Li et al. [2017], ProxEmbed Liu et al. [2017], and PathRank Yang et al.”
BackgroundMethodInfluentialFrom Recognition to Prediction: Analysis of Human Action and Trajectory Prediction in VideoPreprint
Liang, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.10670 · Cited by 2
Multi-stage Optimization based Adversarial TrainingPreprint
Wang, X., Song, C., Wang, L., & He, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.15357 · Cited by 6
“Recalling that multi-stage optimization has been investigated in curriculum learning [4] and transfer learning [3], where the training process is divided into several sub-tasks to be completed stage by stage.”
BackgroundPrioritized training on points that are learnable, worth learning, and not yet learned.Preprint
Mindermann, S., Razzak, M., Xu, W., Kirsch, A., Sharma, M., Morisot, A., Gomez, A. N., Farquhar, S., Brauner, J., & Gal, Y. (2021) · arXiv (Cornell University) · Cited by 2
When does loss-based prioritization fail?Preprint
Hu, N. T., Hu, X., Liu, R., Hooker, S., & Yosinski, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.07741 · Cited by 4
“Curriculum learning (Bengio et al., 2009) devised a strategy of supplying easy and prototypical examples first and gradually increases their difficulty, which has shown to be beneficial to the overall generalization of the model.”
BackgroundFollow Your Path: a Progressive Method for Knowledge DistillationPreprint
Shi, W., Song, Y., Zhou, H., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.09305 · Cited by 2
“One reasonable explanation beyond the above facts could be derived from the perspective of curriculum learning [3]: the learning process will be boosted if the goal is set suitable to the underlying learning preference (bias).”
BackgroundMimicBot: Combining Imitation and Reinforcement Learning to win in Bot BowlPreprint
Pezzotti, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.09478 · Cited by 2
“Reinforcement Learning using A2C (Mnih et al., 2016) with Curriculum Learning (Bengio et al., 2009) was previously applied in the Bot Bowl II competition, but the resulting bot could win only 70% of the time against a random bot, while it could not beat a scripted bot that has roughly the performance of a novice human…”
MethodXiao, C., Lu, P., & He, Q. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.12869 · Cited by 2
“Curriculum learning [20] is a training technique that divides the training process into a sequence of subtasks with increased difficulty levels, which is known to be able to improve the convergence by letting the agent learn on a simplified problem at the beginning stage [21].”
Background
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.