Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 2,401–2,500 of 5,240
Reducing 0s bias in video moment retrieval with a circular competence-based captioner
Wang, G., Wu, X., Liu, Z., & Qin, Z. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103147 · Cited by 7
Curriculum-Style Fine-Grained Adaption for Unsupervised Cross-Lingual Dependency Transfer
Guo, P., Huang, S., Jiang, P., Sun, Y., Zhang, M., & Zhang, M. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3224302 · Cited by 6
“Curriculum Learning has been proposed since very early [44], [45], [46], which is inspired by the learning process of human beings.”
BackgroundExploring the Open World Using Incremental Extreme Value Machines
Koch, T., Liebezeit, F., Rieß, C., Christlein, V., & Köhler, T. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956423 · Cited by 8
“Curriculum and self-paced learning are concepts to sequentially incorporate samples into a model in a meaningful order [21]– [23].”
An, P., Liang, J., Hong, X., Quan, S., Ma, T., Chen, Y., Wang, L., & Ma, J. (2023) · Remote Sensing · doi:10.3390/rs15030627 · Cited by 8
A reinforcement learning approach to system modularization under constraints
Sanaei, R., Otto, K., Hölttä‐Otto, K., et al. (2023) · Systems Engineering · doi:10.1002/sys.21666 · Cited by 7
“For further knowledge of theoretical foundations of this approach please see the paper by Bengio et. al.”
MethodDeep Reinforcement Learning for Automated Car Parking
Lazzaroni, L., Bellotti, F., Capello, A., et al. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-3-031-30333-3_16 · Cited by 7
Towards Practical Few-shot Federated NLP
Cai, D., Wu, Y., Yuan, H., Wang, S., Lin, F. X., & Xu, M. (2023) · Workshop on Machine Learning and Systems (EuroMLSys) · doi:10.1145/3578356.3592575 · Cited by 16
“Excessive inference for pseudo labeling Pacing [23, 33], early-exit [34, 35].”
BackgroundEvidence Reasoning and Curriculum Learning for Document-level Relation Extraction
Xu, T., Qu, J., Hua, W., Li, Z., Xu, J., Liu, A., Zhao, L., & Zhou, X. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3292974 · Cited by 9
“[25] firstly introduces the definition of curriculum learning that “starting small or easy” strategies can benefit machine learning algorithms.”
BackgroundYiğit, G., & Amasyalı, M. F. (2023) · Knowledge and Information Systems · doi:10.1007/s10115-023-01918-2 · Cited by 7
“• Weachieved improvedperformances on the proposed technique especiallywhen adopting Curriculum Learning strategy by considering double and triple sequences of RACE, OBQA, and ARC-Easy.”
BackgroundInfluentialLi, J., Sun, J., Li, X., Yang, Y., Jiang, X., & Li, R. (2023) · Journal of Marine Science and Engineering · doi:10.3390/jmse11071388 · Cited by 8
Corruption depth: Analysis of DNN depth for misclassification
Agarwal, A., Vatsa, M., Singh, R., & Ratha, N. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.11.035 · Cited by 8
Hicks, C., Mavroudis, V., Foley, M., Davies, T., Highnam, K., & Watson, T. (2023) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3605764.3623986 · Cited by 23
“Curriculum learning (CL) [9] is an approach that can help agents learn more effectively by first training on simpler tasks and then gradually increasing the complexity.”
BackgroundBejan, I., Sokolov, A., & Filippova, K. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.625 · Cited by 7
Deeply Supervised Block-Wise Neural Architecture Search
Yang, A., Liu, Y., Li, C., & Ren, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3347542 · Cited by 11
FDC-NeRF: Learning Pose-Free Neural Radiance Fields with Flow-Depth Consistency
Gao, H., Shen, S., Zhang, Z., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446550 · Cited by 7
“We draw inspiration from curriculum learning [12] and introduce the Adaptive Pose-Aware Sampling (APAS) strategy.”
BackgroundMethodSPContrastNet: A Self-Paced Contrastive Learning Model for Few-Shot Text Classification
Chen, J., Zhang, R., Jiang, X., & Hu, C. (2024) · ACM Transactions on Information Systems · doi:10.1145/3652600 · Cited by 16
“Self-paced learning [34] extends curricular learning [4] by attempting to optimize the model more efficiently with easy examples in the earlier training stage and more difficult examples in the latter training stage.”
BackgroundCurriculum pre-training for stylized neural machine translation
Zou, A., Wu, X., Li, X., Zhang, T., Cui, F., & Xu, J. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05586-9 · Cited by 9
“Bengio et al. [18] propose the concept of curriculum learning and apply it to machine learning, which has been shown to have better generalization ability and faster convergence rate [19].”
Interactive Learning from Multiple Noisy Labels
Vembu, S., & Zilles, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46128-1_31 · Cited by 7
“This setting is also known as curriculum learning [1, 2, 3] or self-paced learning [4, 5, 6] in the literature.”
BackgroundScalable Recollections for Continual Lifelong Learning
Riemer, M., Klinger, T., Bouneffouf, D., & Franceschini, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011352 · Cited by 48
“This motivates the automated curriculum learning setting (Bengio et al., 2009) as recently explored for multi-task learning in (Graves et al., 2017) or rather automated generative curriculum learning in our case.”
MethodJang, S., & Han, M. (2018) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc.2018.8539438 · Cited by 9
“Among the several related works, reward shaping [6] to tackle the spare and delayed reward and curriculum learning [7] that gradually increases the difficulties of the problem to better train agents have gained high attentions.”
BackgroundLearning to learn with active adaptive perception
Bossens, D. M., Townsend, N., & Sobey, A. J. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.006 · Cited by 11
“…are heavily inspired by reinforcement learning with intrinsic motivation (Storck et al., 1995; Schmidhuber, 1991; Singh et al., 2004), lifelong reinforcement learning (Bengio et al., 2009; Thrun & Mitchell, 1995; Silver et al., 2013; Ring, 1994), and developmental psychology, e.g. (Pia-get, 1952).”
BackgroundProgressive Latent Models for Self-Learning Scene-Specific Pedestrian Detectors
Ye, Q., Zhang, T., & Ke, W. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2911315 · Cited by 7
“Inspired by the cognitive principle of humans, Bengio et al. [47] proposed self-paced learning (SPL), where a model was learned by gradually including samples from easy to complex.”
MethodLearning a Behavioral Repertoire from Demonstrations
Justesen, N., González-Duque, M., Cabarcas, D., Mouret, J.-B., & Risi, S. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231897 · Cited by 7
Self-Imitation Learning of Locomotion Movements through Termination Curriculum
Babadi, A., Naderi, K., & Hämäläinen, P. (2019) · Motion in Games · doi:10.1145/3359566.3360072 · Cited by 10
“Curriculum learning is the process of learning a series of tasks in increasing order of complexity [4].”
BackgroundPrivacy-Preserving Stochastic Gradual Learning
Han, B., Tsang, I. W.-H., Xiao, X., Chen, L., Fung, S.-F., & Yu, C. P. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2963977 · Cited by 9
“Our inspiration springs from the learning process of CL, namely learning from “easy” tasks to “complex” tasks, which is often used for training robust models [29,30].”
BackgroundA light neural network for modulation detection under impairments
Courtat, T., & Bourboux, H. D. M. D. (2021) · International Symposium on Networks, Computers and Communications (ISNCC) · doi:10.1109/isncc52172.2021.9615851 · Cited by 7
“Following the methods of curriculum learning [27], only few epochs are needed to perform this re-training, if the weights are initialized to the best values found when trained on the simpler Augmod data set.”
MethodEntity Resolution in Dynamic Heterogeneous Networks
Shekhar, S., Pai, D., & Ravindran, S. (2020) · The Web Conference (WWW) · doi:10.1145/3366424.3391264 · Cited by 7
“Following [3, 42], we set up a curriculum training scheme such that initial training iterations use the common negative examples set, and in subsequent iterations we increase number of node specific negative examples.”
MethodKong, W., Zhou, D., Zhang, K., Yang, Z. C., & Yang, W. (2020) · International Conference on Machine Learning and Applications · doi:10.1109/icmla51294.2020.00238 · Cited by 7
“The initial task is used to guide the learner so that the learner can better complete the final complex task [23].”
MethodProrok, A., Blumenkamp, J., Li, Q., Kortvelesy, R., Liu, Z., & Stump, E. (2022) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/mtwp6998 · Cited by 14
SPACE: A Simulator for Physical Interactions and Causal Learning in 3D Environments
Nanyang, J. D., Yu, S., & Tan, C. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00233 · Cited by 10
“The purpose of the SPACE dataset is to serve as the building blocks for curriculum learning [4] in more complex real-world object interaction tasks.”
BackgroundMethodInfluentialvon Bülow, F., & Meisen, T. (2022) · PHM Society European Conference · doi:10.36001/phme.2022.v7i1.3312 · Cited by 9
Neural Decoder for Topological Codes using Pseudo-Inverse of Parity Check Matrix
Chinni, C., Kulkami, A., Pai, D. M., Mitra, K., & Sarvepalli, P. K. (2019) · IEEE Information Theory Workshop (ITW) · doi:10.1109/itw44776.2019.8989133 · Cited by 11
“This progressive training is similar to the common practice of curriculumlearning in neural networks so that the optimizer converges to a better local minima [30].”
MethodMask-Dependent Phase Estimation for Monaural Speaker Separation
Ni, Z., & Mandel, M. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054265 · Cited by 7
“Inspired by [19], we also apply a curriculum training strategy [20] that trains the network using α = 0.”
MethodLong-Term Image Boundary ExtrapolationPreprint
Bhattacharyya, A., Malinowski, M., Schiele, B., et al. (2016) · arXiv (Cornell University) · Cited by 7
Between-class Learning for Image ClassificationPreprint
Tokozume, Y., Ushiku, Y., & Harada, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10284 · Cited by 7
“Furthermore, we assume that the usage of curriculum learning [4] would be helpful to speed up the training; namely, at the early stage, we generate a mixing ratio close to 0 or 1 and input relatively pure examples, and we gradually change the distribution of r to flat.”
BackgroundSelf-Training Ensemble Networks for Zero-Shot Image Recognition.Preprint
Ye, M., & Guo, Y. (2018) · arXiv (Cornell University) · Cited by 6
Neural Arithmetic Expression CalculatorPreprint
Chen, K., Dong, Y., Qiu, X., & Chen, Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.08590 · Cited by 11
“[12] brings general interests to curriculum learning.”
Towards Understanding Generalization via Analytical Learning TheoryPreprint
Kawaguchi, K., Benigo, Y., Verma, V., & Kaelbling, L. P. (2018) · arXiv (Cornell University) · Cited by 6
Kuse, M., & Shen, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.06962 · Cited by 7
“[54], it was observed that for faster convergence, it is crucial to select triplets from the training dataset, that violate the triplet constraint, ie.”
BackgroundAdaptation Across Extreme Variations using Unlabeled Domain BridgesPreprint
Dai, S., Sohn, K., Tsai, Y., Carin, L., & Chandraker, M. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02238 · Cited by 13
“This implies that it is important to know the curriculum [4] (i.e., adaptation difficulty) and our proposed method with multiple discriminators with bridge to target adaptation objective effectively utilizes such information at training.”
MethodFeng, J., Tao, C., Wu, W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04413 · Cited by 7
“Previous studies including curriculum learning (CL) (Bengio et al., 2009) and self-paced learning (SPL) (Jiang et al., 2014, 2015) tackle the problem with heuristics, such as ordering data from easy instances to hard ones (Spitkovsky et al., 2010; Tsvetkov et al., 2016) and retaining training…”
BackgroundBeyond Error Propagation in Neural Machine Translation: Characteristics of Language Also MatterPreprint
Wu, L., Tan, X., He, D., Tian, F., Qin, T., Lai, J., & Liu, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00120 · Cited by 13
“In this section, we study different branching languages from two aspects, including the n-gram statistics of a target language, which has been used as a kind of characterization of hardness of learning (Bengio et al., 2009), and the dependency statistics in parse trees.”
BackgroundDetector Discovery in the Wild: Joint Multiple Instance and Representation LearningPreprint
Hoffman, J., Pathak, D., Darrell, T. J., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1135 · Cited by 7
“Approaches to better optimize the non-convex problem involve using multifold learning as a measure of regularizing overfitting [8], optimize Latent SVM for the area under the ROC curve (AUC) [6] and training with easy examples in beginning to avoid bad local optimization [5, 24].”
MethodAdversarial Learning of General Transformations for Data AugmentationPreprint
Mounsaveng, S., Vázquez, D., Ayed, I. B., & Pedersoli, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09801 · Cited by 11
“We believe that for good performance in data augmentation it is not just about generating plausible augmented samples, but also about generating the right samples at the right moment, as in curriculum learning [2].”
BackgroundCompound Probabilistic Context-Free Grammars for Grammar InductionPreprint
Kim, Y., Dyer, C., & Rush, A. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.10225 · Cited by 7
“We employ a curriculum learning strategy (Bengio et al., 2009) where we train only on sentences of length up to 30 in the first epoch, and increase this length limit by 1 each epoch.”
MethodTask Selection Policies for Multitask LearningPreprint
Glover, J. D., & Hokamp, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.06214 · Cited by 9
“One way to approach the question of how much training budget to spend on each task is to view this as a curriculum learning problem (Elman, 1993; Bengio et al., 2009).”
BackgroundCurriculum based Dropout Discriminator for Domain AdaptationPreprint
Kurmi, V. K., Bajaj, V., Subramanian, V. K., & Namboodiri, V. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.10628 · Cited by 13
“The curriculum learning [2] enhances model’s performance and its generalization capability.”
BackgroundMethodTeacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark DetectionPreprint
Dong, X., & Yang, Y. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.02116 · Cited by 7
“Others exploit pseudo-labels of unlabeled data in the semi-supervised scenario [1, 2, 23, 28].”
BackgroundHJB Optimal Feedback Control with Deep Differential Value Functions and Action ConstraintsPreprint
Lutter, M., Belousov, B., Listmann, K. D., Clever, D., & Peters, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06153 · Cited by 12
“This learning scheme of changing the parameter ρ during optimization can be interpreted as continuation method [17] and curricular learning [18].”
MethodA Simple General Approach to Balance Task Difficulty in Multi-Task LearningPreprint
Liang, S., & Zhang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04792 · Cited by 14
“Curriculum learning (Bengio et al., 2009) and its variant self-paced learning (Kumar et al., 2010), aim to solve non-convex objective functions by firstly learning from easy data points and then from harder ones.”
BackgroundAttentive Fusion Enhanced Audio-Visual Encoding for Transformer Based Robust Speech RecognitionPreprint
Wei, L., Zhang, J., Hou, J., & Dai, L.-R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.02686 · Cited by 24
“During the pre-training phase, we introduce the curriculum learning (CL) [26] in the first 4 epochs.”
MethodZhang, C., Li, Y., Chen, X., Jin, Y., Tang, P., & Li, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01265 · Cited by 6
“uct an ensemble model with Ksub-models, each of which uses the 30% to 100% of the easiest samples (the samples with lowest losses), which can be regarded as an ensemble version of curriculum learning [28]. Performance metrics Precision: While standard classification problems care about the prediction accuracy on all th…”
InfluentialWei, J., Suriawinata, A. A., Ren, B., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.13698 · Cited by 7
“One of the earliest works demonstrating the benefit of curriculum learning [1] posits that learning occurs better when examples are not randomly presented but instead organized in a meaningful order that gradually shows more concepts and complexity.”
BackgroundInfluentialA Survey on Deep Learning and Explainability for Automatic Image-based Medical Report GenerationPreprint
Messina, P., Pino, P., Parra, D., Soto, Á., Besa, C., Uribe, S. A., Andía, M. E., Cristián Tejos, N., Prieto, C., & Daniel Capurro, N. (2020) · arXiv (Cornell University) · Cited by 7
Robust Quadrupedal Locomotion on Sloped Terrains: A Linear Policy ApproachPreprint
Paigwar, K., Krishna, L., Tirumala, S., Khetan, N., Sagi, A., Joglekar, A., Bhatnagar, S., Ghosal, A., Amrutur, B. S., & Kolathaya, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.16342 · Cited by 20
“We are using the curriculum learning technique [31] for training the policy which makes the learning smooth, efficient, and effective.”
MethodA Model of Two Tales: Dual Transfer Learning Framework for Improved Long-tail Item RecommendationPreprint
Zhang⋆, Y., Cheng, D. Z., Yao, T., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.15982 · Cited by 7
“large amount of tail items and we further enhanced the metamapping to tail items by learning connections among items via curriculum learning. Curriculum Learning. The curriculum learning, proposed by [4], has shown good performance in different areas [15, 19]. The basic idea is to simulates the humans learning process…”
BackgroundPrivileged Knowledge Distillation for Online Action DetectionPreprint
Zhao, P., Xie, L., Zhang, Y., Wang, Y., & Tian, Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.09158 · Cited by 25
“Curriculum Learning [4] is proposed by Bengio et al. that teaches a model starting from easy samples to hard samples.”
BackgroundWhen Do Curricula Work?Preprint
Wu, X., Dyer, E., & Neyshabur, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03107 · Cited by 92
“This is in tension with experiments in other contexts, however, which demonstrate that anti-curricula under perform standard or curriculum training (Bengio et al., 2009; Hacohen & Weinshall, 2019).”
BackgroundMethodResultInfluentialSinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-TrainingPreprint
Tai, K. S., Bailis, P. D., & Valiant, G. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.08622 · Cited by 60
“This assignment strategy is reminiscent of curriculum learning (Bengio et al., 2009) and self-paced learning (Kumar et al., 2010), where “easy” examples are used early in training and more “difficult” examples are gradually introduced over time.”
BackgroundMethodNPT-Loss: A Metric Loss with Implicit Mining for Face RecognitionPreprint
Khalid, S. S., Awais, M., Chan, C.-H., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03503 · Cited by 7
“In [31], an effort is made to combine the advantages of margin and feature mining and recently in [13], curriculum-learning [1] is incorporated with ArcFace to improve the performance.”
MethodA Realistic Evaluation of Semi-Supervised Learning for Fine-Grained ClassificationPreprint
Su, J.-C., Cheng, Z., & Maji, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.00679 · Cited by 7
“The pseudo-labels can be added iteratively to induce a “curriculum” [4,8,17].”
BackgroundNon-Linear Fusion for Self-Paced Multi-View ClusteringPreprint
Huang, Z., Ren, Y., Pu, X., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.09255 · Cited by 7
Manipulating SGD with Data Ordering AttacksPreprint
Shumailov, I., Shumaylov, Z., Kazhdan, D., Zhao, Y., Papernot, N., Erdogdu, M. A., & Anderson, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.09667 · Cited by 111
“We consider four reordering policies, motivated by research in the fields of curriculum learning [5] and core-set selection [2], which discovered that model training can be enhanced by scheduling how and what data is presented to the model.”
BackgroundToAlign: Task-oriented Alignment for Unsupervised Domain AdaptationPreprint
Wei, G., Lan, C., Zeng, W., Zhang, Z., & Chen, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.10812 · Cited by 78
“CMSS [66] exploits Curriculum Learning (CL) [4] to align target samples with the dynamically selected source samples to exploit the different transferability of the source samples.”
BackgroundMonitoring hydrodynamic vessel performance by incremental machine learning using in-service data
Mittendorf, M., Nielsen, U. D., & Gundermann, D. (2024) · Ship Technology Research · doi:10.1080/09377255.2024.2362012 · Cited by 6
“Bengio et al. (2009) provide a theoretical background for curriculum learning.”
Akujuobi, U., Kumari, P., Choi, J., Badreddine, S., Maruyama, K., Palaniappan, S. K., & Besold, T. R. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10885-1 · Cited by 11
“Inspired by human learning, Bengio et al. (2009) introduced the concept of progressive training, wherein the model is trained on increasingly difficult training samples.”
Ensembling Prioritized Hybrid Policies for Multi-agent Pathfinding
Tang, H., Berto, F., & Park, J. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801914 · Cited by 7
“Curriculum learning [38], which is shown to be effective in previous works [14], [17], is also adopted.”
Motif-aware curriculum learning for node classification
Cai, X., Chen, M.-S., Wang, C., & Zhang, H. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.107089 · Cited by 6
Zhao, L., Zhou, Y., Zhong, W., Jin, C., Liu, B., & Li, F. (2025) · Remote Sensing · doi:10.3390/rs17020277 · Cited by 8
CurMIM: Curriculum Masked Image Modeling
Liu, H., Wang, K., Han, Y., Wang, H., Hu, Y., Wang, C., & Nie, L. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890877 · Cited by 8
“The human learning behavior, which is also known as “curriculum learning”, is a training strategy where tasks are learned in increasing order of difficulty [21].”
Hierarchical residual reinforcement learning based path tracking control method for unmanned bicycle
Huo, B., Yu, L., Liu, Y., & Chen, Z. (2025) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2025.104996 · Cited by 9
Reinforcement learning for evaluating school safety designs in active shooter incidents
Liu, R., Wu, W., Becerik-Gerber, B., Lucas, G., Laboy, M. M., & Fannon, D. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103575 · Cited by 10
Budiarjo, T., Pradata, S. Y., Santiyuda, G., Amrizal, M. A., Pulungan, R., & Takizawa, H. (2025) · doi:10.1145/3718350.3718359 · Cited by 11
Wang, S., Zhao, H. Q., Shu, T., Pan, Z., Liang, G., & Zhao, J. (2025) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2025.3593340 · Cited by 9
“Curriculum learning (CL) enables agents to progress from simpler to more complex tasks [28], which has been shown to accelerate convergence, improve generalization, and enhance exploration, particularly in complex tasks with sparse rewards or high-dimensional action spaces [29].”
Learning Noise-Robust Stable Koopman Operator for Control With Hankel DMD
Sakib, S. A., & Pan, S. (2025) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2025.3600710 · Cited by 6
“Curriculum learning [50] is a training strategy in which a model is first trained in simpler tasks and then gradually exposed to more challenging ones.”
Zhao, X. (2025) · Scientific Reports · doi:10.1038/s41598-025-17187-8 · Cited by 15
“Model training employs a curriculum learning strategy that progressively increases task difficulty 88 .”
Detailed Delineation of the Fetal Brain in Diffusion MRI via Multi-Task Learning
Karimi, D., Calixto, C. A., Snoussi, H., Li, B., Cortés-Albornoz, M. C., Velasco‐Annis, C., Rollins, C. K., Pierotich, L., Jaimes, C. E., Gholipour, A., et al. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3619809 · Cited by 16
“E ff ective sharing of parameters / activations among the tasks [42], [43], manipulation of the optimization gradients [44], and task scheduling and prioritization [45], [46] are some of the techniques that are widely used to enable MTL in challenging applications.”
Learning to decode logical circuits
Zhou, Y., Wan, C., Xu, Y., Zhou, J. P., Weinberger, K. Q., & Kim, E.-A. (2025) · Nature Computational Science · doi:10.1038/s43588-025-00897-4 · Cited by 15
“Second, it allows us to use a resource-efficient curriculum training strategy 25 .”
What neuroscience can tell AI about learning in continuously changing environments
Durstewitz, D., Averbeck, B. B., & Koppe, G. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01146-z · Cited by 12
“Developmental plasticity windows, finally, are mirrored by strategies and curriculum learning protocols that differentially regulate the plasticity and learning rates of different network layers 179–182 .”
Madduri, M. M., Yamagami, M., Li, S. J., Burckhardt, S. N., Burden, S. A., & Orsborn, A. L. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01194-z · Cited by 10
BAOD: Budget-Aware Object Detection
Pardo, A., Xu, M., Thabet, A., Arbeláez, P., & Ghanem, B. (2021) · LatinX in AI at Computer Vision and Pattern Recognition Conference (LXAI) · doi:10.1109/cvprw53098.2021.00137 · Cited by 17
“er, and J. M. K¨ ¨ohler. The power of ensembles for active learning in image classifi- cation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9368–9377, 2018.3 [3]Y. Bengio, J. Louradour, R. Collobert, and J. Weston. Curriculum learning. In Proceedings of the 26th annual interna…”
BackgroundInfluentialLearning-to-Fly RL: Reinforcement Learning-based Collision Avoidance for Scalable Urban Air Mobility
Jang, K. J., Pant, Y. V., Rodionova, A., & Mangharam, R. (2020) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc50938.2020.9256710 · Cited by 8
“Bengio et al. [19] first applied CL to supervised learning and then extended to RL by Narvekar et al. [20] which presented a mathematical framework for CL with discrete task spaces.”
BackgroundSelf-Supervised Learning for Heterogeneous Audiovisual Scene Analysis
Hu, D., Wang, Z., Nie, F., Wang, R., & Li, X. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3162477 · Cited by 6
“pared with a disordered arrangement, starting from easier samples or tasks then gradually increasing the difficulty level could contribute to better learning performance [44], [45], which is”
BackgroundProgressive Hard Negative Masking: From Global Uniformity to Local Tolerance
Sun, Q., Zhang, W., & Lin, X. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3269795 · Cited by 11
Uncertainty-Aware Few-Shot Class-Incremental Learning
Zhu, J., Zhao, J., Zhou, J., He, L., Yang, J., & Zhang, Z. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096589 · Cited by 11
“Humans’ learning process is simple to difficult, [16, 17] and they can learn new concepts easily.”
BackgroundLearning Pre-Grasp Manipulation of Flat Objects in Cluttered Environments using Sliding Primitives
Wu, J., Wu, H., Zhong, S., Sun, Q., & Li, Y. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160869 · Cited by 7
“Furthermore, curriculum learning (CL) is utilized to extend our method into cluttered environments with more objects, overcoming the exploration problem in reinforcement learning [14].”
MethodXu, X., Li, Y., & Lu, C. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01850-6 · Cited by 9
“Curriculum Learning is proposed by Bengio et al. (2009).”
BackgroundIDO: Instance dual-optimization for weakly supervised object detection
Ren, Z., Tang, Y., & Zhang, W. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04956-z · Cited by 6
“First, the IWSS is designed for instance denoising based on curriculum learning [18] in the training phase.”
MethodCoğrafya Öğretmenlerinin Öğretim Programı Okuryazarlık Düzeylerinin İncelenmesi
DÜNDAR, E., SÖZCÜ, U., & TÜRKER, A. (2023) · Gazi Üniversitesi Gazi Eğitim Fakültesi Dergisi · doi:10.17152/gefad.1301335 · Cited by 6
Revisiting Modality Imbalance In Multimodal Pedestrian Detection
Das, A. K., Das, S. K., Sistu, G., Horgan, J., Bhattacharya, U., Jones, E., Glavin, M., & Eising, C. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10222711 · Cited by 11
“Curriculum Learning [18] was considered it has been helpful to achieve better generalization in our previous works.”
Robust Multitask Learning With Sample Gradient Similarity
Peng, X., Chang, C., Wang, F., & Li, L. (2023) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3315541 · Cited by 11
“For example, the landmark work of curriculum learning [52] proposes that the weights should be dynamically assigned according to the easiness of different training samples.”
BackgroundEnabling Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representation
Nipu, A. S., Liu, S., & Harris, A. (2023) · IEEE Conference on Games (CoG) · doi:10.1109/cog57401.2023.10333236 · Cited by 7
“Curriculum learning is a specific type of transfer learning that arranges a series of tasks according to their increasing level of complexity [31].”
BackgroundDeng, L., Yang, B., Kang, Z., Wu, J., Li, S., & Xiang, Y. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01360-z · Cited by 10
A Curriculum-Based Meta-Learning Approach for Few-Shot Fault Diagnosis
Lai, P. L., Zhang, F., Li, T., Guo, J. T., Teng, F., & Liu, R. (2023) · International Conference on Digital Society and Intelligent Systems (DSInS) · doi:10.1109/dsins60115.2023.10455545 · Cited by 6
“Curriculum learning is a learning paradigm proposed by the famous scholar Bengio [9].”
BackgroundAda, S. E., Say, H., Uğur, E., & Öztop, E. (2024) · IEEE Access · doi:10.1109/access.2024.3402089 · Cited by 7
“…leads to an emergent interleaved multi-task learning regime. c) Curriculum Reinforcement Learning: Curriculum learning methods focus on discovering a goal or task sequencing procedure that can lead to a faster convergence during training or improved performance compared to random sequencing [41].”
BackgroundValue Approximation for Two-Player General-Sum Differential Games With State Constraints
Zhang, L., Ghimire, M., Zhang, W., Xu, Z., & Ren, Y. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3411850 · Cited by 6
“Hence the second solution, called value hardening , following curriculum learning [13], aims to improve the chance of learning the discontinuous boundaries by gradually increasing the Lipschitz constant of a constraint violation penalty.”
BackgroundCrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community
Liu, Y., Guo, B., Li, N., Ding, Y., Zhang, Z., & Yu, Z. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3423319 · Cited by 17
“Curriculum learning was first proposed in 2009, and its core definition is to start learning models from easy samples and gradually advance to complex samples [161], belonging to the evolution mode.”
Efficient Reinforcement Learning for 3D Jumping Monopods
Bussola, R., Focchi, M., Prete, A. D., Fontanelli, D., & Palopoli, L. (2024) · Sensors · doi:10.3390/s24154981 · Cited by 8
“Following the curriculum learning idea [39], we gradually increase the difficulty of the jump enlarging the bounds of the training region (where the targets are sampled) in accordance with the number of episodes and the average reward.”
BackgroundPatient teacher can impart locality to improve lightweight vision transformer on small dataset
Ling, J., Zhang, X., Du, F., Li, L., Shang, W., Gao, C., & Li, T. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110893 · Cited by 8
“Curriculum Learning: Curriculum learning, originally introduced by prior research [29], is a method for training networks by organizing the sequence of learning tasks and incrementally increasing the learning difficulty.”
BackgroundMethodInfluentialDeep Reinforcement Learning With Curriculum Design for Quantum State Classification
Yu, H., & Zhao, X. (2024) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2024.3479202 · Cited by 6
“A meaningful sequence of tasks induces a curriculum [35] as the experiences being transferred from one task to the next one in the learning process, known as curriculum learning [36].”
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.