Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 1,601–1,700 of 5,240
When Broad Learning System Meets Label Noise Learning: A Reweighting Learning Framework
Liu, L., Chen, J., Yang, B., Feng, Q., & Chen, C. L. P. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3317255 · Cited by 17
“More recently, curriculum learning [46], [47] has been developed to deal with imbalanced data or noisy data, which wields the samples from easy to difficult to gradually train the model, imitating the meaningful learning order in human curricula.”
DCL: Dipolar Confidence Learning for Source-Free Unsupervised Domain Adaptation
Tian, Q., Sun, H., Peng, S., Zheng, Y., Wan, J., & Lei, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3332353 · Cited by 17
“It is worth noting that, although our motivation is seemingly similar to self-paced learning [68], [69] or curriculum learning [70], there is significant difference between them in that the self-paced or curriculum learning explores knowledge in order from easy to hard by changing the thresholds,…”
BackgroundLi, Z., Xin, J., & Li, N. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342066 · Cited by 29
“To address the above issues, we propose the following solutions: On the one hand, we apply curriculum learning (CL) [13] to improve sample efficiency and speed up convergence, which starts learning from simple tasks and gradually increases the difficulty of the tasks [11].”
MethodMulti-Robot Coverage Path Planning for the Inspection of Offshore Wind Farms: A Review
Foster, A. J. I., Gianni, M., Aly, A., Samani, H. A., & Sharma, S. K. (2023) · Drones · doi:10.3390/drones8010010 · Cited by 16
Lee, S.-J., Oh, H. J., Son, Y., Kim, J., Kwon, I.-J., Kim, B., Lee, J.-H., & Kim, H.-K. (2024) · BMC Oral Health · doi:10.1186/s12903-024-03898-3 · Cited by 27
“In this study, we introduced an effective technique called "mosaic-based soft labeling" augmentation combined with curriculum learning (CL) [31].”
MethodTeng, X., Shen, X., Xu, K., & Lan, L. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680938 · Cited by 23
“Inspired by the human learning process, curriculum learning is proposed by [2] as a training strategy by learning from easy samples to hard samples.”
Arm-Constrained Curriculum Learning for Loco-Manipulation of a Wheel-Legged Robot
Wang, Z., Jia, Y., Shi, L., Wang, H., Zhao, H., Li, X., Zhou, J., Ma, J., & Zhou, G. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802062 · Cited by 41
“Previous research has highlighted the advantages of employing a curriculum of task difficulty to train complex policies [30], [31].”
AD-VAT: An Asymmetric Dueling mechanism for learning Visual Active Tracking.
Zhong, F., Sun, P., Luo, W., Yan, T., & Wang, Y. (2018) · International Conference on Learning Representations · Cited by 34
Local to Global Learning: Gradually Adding Classes for Training Deep Neural Networks
Cheng, H., Lian, D., Deng, B., Gao, S., Tan, T., & Geng, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00488 · Cited by 14
“Curriculum learning (CL) was first introduced in 2009 by Bengio et al [3].”
BackgroundEvaluating the progress of deep learning for visual relational concepts
Stabinger, S., Peer, D., Piater, J., & Rodríguez-Sánchez, A. J. (2021) · Journal of Vision · doi:10.1167/jov.21.11.8 · Cited by 24
Curriculum Enhanced Supervised Attention Network for Person Re-Identification
Zhu, X., Qian, J., Wang, H., & Liu, P. (2020) · IEEE Signal Processing Letters · doi:10.1109/lsp.2020.3024794 · Cited by 17
“On the other hand, we take deep insight into curriculum learning [17], design a multi-stage training strategy to improve the capability of the network evolutionarily and resist the impact from noisy labels.”
BackgroundSelf-Induced Curriculum Learning in Self-Supervised Neural Machine Translation
Ruiter, D., van Genabith, J., & España-Bonet, C. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.202 · Cited by 20
“This approach to learning based on sampling from a curriculum of increasing complexity has also been shown to be beneficial for machines and is referred to as curriculum learning (CL) (Bengio et al., 2009).”
BackgroundInfluentialDeep Reinforcement Learning for Humanoid Robot Dribbling
Muzio, A. F. V., Máximo, M. R. O. A., & Yoneyama, T. (2020) · Latin American Robotics Symposium, Brazilian Symposium on Robotics and Workshop on Robotics in Education (LARS/SBR/WRE) · doi:10.1109/lars/sbr/wre51543.2020.9307084 · Cited by 16
A Reinforcement Learning Framework for QoS-Driven Radio Resource Scheduler
Shekhawat, J. S., Agrawal, R., Shenoy, K. G., & Shashidhara, R. (2020) · Globecom · doi:10.1109/globecom42002.2020.9322182 · Cited by 20
“We employ curriculum learning [15] to accelerate convergence of RL agents using episodic MDP.”
MethodTemporal LiDAR Frame Prediction for Autonomous Driving
Deng, D., & Zakhor, A. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00093 · Cited by 17
“There exist two popular approaches for training generative, temporal networks: curriculum approaches [3] and teacher-forcing [9].”
BackgroundPMT-Net: Progressive Multi-Task Network for one-shot Person Re-Identification
Zhang, Y., Ma, B., Feng, Y., & Li, M. (2021) · Information Sciences · doi:10.1016/j.ins.2021.03.048 · Cited by 18
RNN Transducer Models for Spoken Language Understanding
Thomas, S., Kuo, H.-K. J., Saon, G., Tüske, Z., Kingsbury, B. E. D., Kurata, G., Kons, Z., & Hoory, R. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414029 · Cited by 19
“This step is a form of curriculum learning [7, 25] that gradually modifies an off-the-shelf ASR model into a domain-specific SLU model.”
BackgroundEfficient Reinforcement Learning for StarCraft by Abstract Forward Models and Transfer Learning
Liu, R.-Z., Guo, H., Ji, X., Yu, Y., Pang, Z.-J., Xiao, Z., Wu, Y., & Lü, T. (2021) · IEEE Transactions on Games · doi:10.1109/tg.2021.3071162 · Cited by 29
“Curriculum learning (CL) [18], [19] is a learning process that allows learning to grow incrementally through a series of tasks, e.g., repeatedly transfer learning from easier task to harder task.”
Meta-Optimization of Bias-Variance Trade-Off in Stochastic Model Learning
Aotani, T., Kobayashi, T., & Sugimoto, K. (2021) · IEEE Access · doi:10.1109/access.2021.3125000 · Cited by 16
“As in curriculum learning [50], adaptive changes in the optimization target sometimes provide opportunity to escape from the local solutions, but vice versa.”
BackgroundCombining Curriculum Learning and Knowledge Distillation for Dialogue Generation
Zhu, Q., Chen, X., Wu, P., Liu, J., & Zhao, D. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.111 · Cited by 42
“Xing, 2016; Feng et al., 2019) employ curriculum learning (Bengio et al., 2009), in which a model is taught by using easy samples firstly and gradually adding more difficult ones.”
BackgroundInfluentialAn Ensemble Model for Combating Label Noise
Lu, Y., Yang, B., & He, W. (2022) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3488560.3498376 · Cited by 20
“Inspired from human cognition, Curriculum learning (CL) [3] proposes to start from easy samples and go through harder samples to improve convergence and generalization.”
Extreme Zero-Shot Learning for Extreme Text Classification
Xiong, Y., Chang, W.-C., Hsieh, C., Yu, H., & Dhillon, I. S. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.399 · Cited by 28
“This approach allows the encoder to learn from the coarse scale to the fine scale of clustering assignment, and is similar to the idea of curriculum learning (Bengio et al., 2009) to first focus on learning from a subset of simple examples, and expanding to include the remaining harder samples.”
Self-Supervised Learning of Audio Representations From Audio-Visual Data Using Spatial Alignment
Wang, S., Politis, A., Mesaros, A., & Virtanen, T. I. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3180592 · Cited by 22
“To solve this audio-visual spatial alignment task, the authors also adopted a curriculum learning strategy [26] by dividing the task into two stages.”
MethodClustering-based Curriculum Construction for Sample-Balanced Federated Learning
Qi, Z., Wang, Y., Chen, Z., Wang, R., Meng, X., & Meng, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20503-3_13 · Cited by 35
“Curriculum learning (CL) imitates the human learning process, which ranks all instances based on complexity, and adopts the knowledge learning method of easy first and then hard [15].”
BackgroundCurriculum learning for data-driven modeling of dynamical systems
Bucci, M. A., Semeraro, O., Allauzen, A., Chibbaro, S., & Mathelin, L. (2023) · The European Physical Journal E · doi:10.1140/epje/s10189-023-00269-8 · Cited by 22
Semi-supervised transfer learning with hierarchical self-regularization
Li, X., Abuduweili, A., Shi, H., Yang, P., Dou, D., Xiong, H., & Xu, C. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109831 · Cited by 17
Language Models as Representations for Weakly-Supervised NLP Tasks
Huang, F., Yates, E., Ahuja, A., & Downey, D. (2012) · Cited by 16
Unsupervised Object Discovery and Segmentation in Videos
Schulter, S., Leistner, C., Roth, P. M., et al. (2013) · British Machine Vision Conference (BMVC) · doi:10.5244/c.27.53 · Cited by 16
“Lee and Grauman [20] apply curriculum learning [4] in order to discover visual categories.”
BackgroundLow-latency Deep Clustering for Speech Separation
Wang, S., Naithani, G., & Virtanen, T. I. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683437 · Cited by 16
“200 (800 ms). We first train the network for 100 frame sequences and then after convergence continue training with 200 frame sequences, known as curriculum learning used in [8] and first introduced in [23]. The idea is to pre-train a network on an easier task first improves learning and generalization. Finally, c) is stu…”
BackgroundDescriptor Selection Improvements for Quantitative Structure-Activity Relationships
Xia, L.-Y., Wang, Q., Cao, Z., et al. (2019) · International Journal of Neural Systems · doi:10.1142/s0129065719500163 · Cited by 16
Label-Similarity Curriculum Learning
Doǧan, Ü., Deshmukh, A. A., Machura, M., & Igel, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58526-6_11 · Cited by 24
“In [1], there is a sequence of weights associated with each example i = 1, .”
BackgroundMethodResultInfluentialTowards Multi Robot Task Allocation and Navigation using Deep Reinforcement Learning
Elfakharany, A., Yusof, R., & Ismail, Z. H. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1447/1/012045 · Cited by 16
“In that case using curriculum learning [13] in which the model is trained on a simpler version of the task first then its trained on the required task can aid the network to learn both tasks.”
MethodHuman Attribute Recognition— A Comprehensive Survey
Yaghoubi, E., Khezeli, F., Borza, D. L., Kumar, S. A., Neves, J. C., & Proença, H. P. M. C. (2020) · Applied Sciences · doi:10.3390/app10165608 · Cited by 24
Exploration in Relational Worlds
Lang, T., Toussaint, M., & Kristian Kersting, B. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15883-4_12 · Cited by 16
“Beyond that, again in contrast to the propositional explorers, the relational explorers are able to transfer the learned knowledge from simple to difficult tasks in the sense of curriculum learning [1], answering the second part of question Q3.”
ResultTay, Y., Wang, S., Tuan, L. A., Fu, J., Phan, M. C., Yuan, X., Rao, J., Hui, S. C., & Zhang, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10847 · Cited by 15
“Our work draws inspiration from curriculum learning (CL) (Bengio et al., 2009).”
BackgroundInfluentialCompetence-based Curriculum Learning for Neural Machine TranslationPreprint
Platanios, E. A., Stretcu, O., Neubig, G., Póczos, B., & Mitchell, T. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.09848 · Cited by 117
“Bengio et al. (2009) propose a curriculum learning approach that aims to address these issues by changing the mini-batch sampling strategy.”
BackgroundMethodInfluentialDecoupled Certainty-Driven Consistency Loss for Semi-supervised LearningPreprint
Liu, L., Li, Y., & Tan, R. T. (2019) · arXiv (Cornell University) · Cited by 15
FQ-Conv: Fully Quantized Convolution for Efficient and Accurate InferencePreprint
Verhoef, B.-E., Laubeuf, N., Cosemans, S., Debacker, P., Papistas, I. A., Mallik, A., & Verkest, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.09356 · Cited by 25
“Gradual quantization is akin to curriculum learning (Bengio et al., 2009”
MethodA Survey on Document-level Neural Machine Translation: Methods and EvaluationPreprint
Maruf, S., Saleh, F., & Haffari, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.08494 · Cited by 34
“Most recently, Stojanovski and Fraser [105] introduced a curriculum learning approach [8] that leverages oracle information to promote anaphora resolution while training the context-aware NMT model.”
BackgroundDeep Audio-Visual Learning: A SurveyPreprint
Zhu, H., Luo, M., Wang, R., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.04758 · Cited by 16
“The researchers applied the curriculum learning strategy [118] to this task and divided the samples into four categories: positives (the corresponding audio-video pairs), easy negatives (audio and video clips originating from different videos), difficult negatives (audio and video clips originating from the same video…”
MethodDisentangling User Interest and Popularity Bias for Recommendation with Causal EmbeddingPreprint
Zheng, Y., Gao, C., Li, X., He, X.-N., Li, Y., & Jin, D.-P. (2020) · arXiv.org · doi:10.48550/arxiv.2006.11011 · Cited by 19
“Inspired by curriculum learning [6], we adopt an easy-to-hard strategy on training DICE by adding decay on margin values and loss weights.”
MethodGradual Fine-Tuning for Low-Resource Domain AdaptationPreprint
Xu, H., Ebner, S., Yarmohammadi, M., White, A. S., Durme, B. V., & Murray, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.02205 · Cited by 31
Fast Multi-Resolution Transformer Fine-tuning for Extreme Multi-label Text ClassificationPreprint
Zhang, J., Chang, W.-C., Yu, H., & Dhillon, I. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.00685 · Cited by 118
“Motivated by the multi-resolution learning in image generation [14–16] and curriculum learning [17], we formulate the XMC problem as a series of sub-problems with multi-resolution label signals and recursively fine-tune the pre-trained transformer on the coarse-to-fine objectives.”
MethodEfficient Vehicular Collaborative Perception Based on Saptial-Temporal Feature Compression
Zhang, J., Yang, K., Wang, H., Sun, P., & Song, L. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3403263 · Cited by 20
“…same time, in order to enable the model to handle different communication conditions and to be trained stably, we apply a curriculum learning strategy [39] in training stage. we make the training process start with a higher communication volume, and gradually decrease the volume of communication.”
MethodHu, Y., Zhou, D. X., Yao, W., Shao, X., & Sun, G. (2025) · Aerospace Science and Technology · doi:10.1016/j.ast.2025.110540 · Cited by 15
Detecting Arbitrary Oriented Text in the Wild with a Visual Attention Model
Huang, W., He, D., Yang, X., Zhou, Z., Kifer, D., & Giles, C. L. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2967282 · Cited by 14
“To solve this problem, we adapted the curriculum learning strategy [5] that we start with easy examples: The model starts training with data samples of sequence length 2.”
MethodSun, Y., Lai, J., Cao, L., Chen, X., Xu, Z., & Xu, Y. (2020) · IEEE Access · doi:10.1109/access.2020.3011670 · Cited by 16
Iterative confidence relabeling with deep ConvNets for organ segmentation with partial labels
Petit, O., Thome, N., & Soler, L. (2021) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2021.101938 · Cited by 18
“This approach could be seen as a curriculum learning strategy [3] or self-paced learning [26], where the easy examples have the most confident predictions and the hard examples have the least confident ones.”
Learning from interpretation transition using differentiable logic programming semantics
Gao, K., Wang, H., Cao, Y., & Inoue, K. (2021) · Machine Learning · doi:10.1007/s10994-021-06058-8 · Cited by 23
“During the learning processes, we reduce the search space and boost the learning speed through curriculum learning: the sys-tem consolidates what it learns in one episode, stores it as background knowledge, and reuses it in subsequent episodes (Bengio et al. 2009).”
BackgroundThe Trade-offs of Domain Adaptation for Neural Language Models
Grangier, D., & Iter, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.264 · Cited by 20
“This gradual adaptation of neural network is related to curriculum learning (Bengio et al., 2009) which studies the ordering of examples and tasks during model training.”
Follow Your Path: A Progressive Method for Knowledge Distillation
Shi, W., Song, Y., Zhou, H., Li, B., & Li, L. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86523-8_36 · Cited by 24
“One reasonable explanation beyond the above facts could be derived from the perspective of curriculum learning [3]: the learning process will be boosted if the goal is set suitable to the underlying learning preference (bias).”
BackgroundLess is More: Adaptive Curriculum Learning for Thyroid Nodule Diagnosis
Gong, H., Cheng, H., Xie, Y., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16440-8_24 · Cited by 15
A framework to build accurate Convolutional Neural Network models for melanoma diagnosis
Pérez, E., & Ventura, S. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110157 · Cited by 20
Real-time route planning of unmanned aerial vehicles based on improved soft actor-critic algorithm
Zhou, Y., Shu, J., Zheng, X., Hao, H., & Song, H. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2022.1025817 · Cited by 17
“Curriculum learning (Bengio et al., 2009) is a training strategy proposed by Bengio.”
BackgroundDisentangling Geographical Effect for Point-of-Interest Recommendation
Qin, Y., Gao, C., Wang, Y., Wei, S., Jin, D., Yuan, J., & Zhang, L. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3221873 · Cited by 24
“Inspired by curriculum learning [30], which first feeds easy training data to the model and then gradually increases the difficulty, the area of regions valid for O 1 is enlarged gradually in the training process.”
MethodNeumann, N. M. P., de Heer, P. B. U. L., & Phillipson, F. (2023) · Quantum Information Processing · doi:10.1007/s11128-023-03867-9 · Cited by 24
“We call the first strategy direct learning and the second strategy curriculum learning [6].”
BackgroundA Review of the Evaluation System for Curriculum Learning
Liu, F., Zhang, T., Zhang, C., Liu, L., Wang, L., & Liu, B. (2023) · Electronics · doi:10.3390/electronics12071676 · Cited by 24
Preference-Aware Group Task Assignment in Spatial Crowdsourcing: Effectiveness and Efficiency
Zhao, Y. L., Liu, J., Li, Y., et al. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3266735 · Cited by 15
“Curriculum Negative Sampling: We propose a Curriculum Negative Sampling (CNS) method, which generates a small amount of negative samples for each worker group, following the principles of curriculum learning [3].”
MethodEnd-to-End Video Scene Graph Generation With Temporal Propagation Transformer
Zhang, Y., Pan, Y., Yao, T., Huang, R., Mei, T., & Chen, C. W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3283879 · Cited by 33
“Training: At the training stage, we adopt a 4-step training scheme to optimize our TPT progressively, which shares a similar spirit of curriculum learning [59].”
MethodChen, H., Wang, X. Y., Lan, X., Chen, H., Duan, X., Jia, J., & Zhu, W. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612504 · Cited by 35
Deep Reinforcement Learning-Based Off-Road Path Planning via Low-Dimensional Simulation
Wang, X., Shang, E., Dai, B., Miao, Q., & Nie, Y. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3347531 · Cited by 21
“6) Adaptive Curriculum Learning: As per prior research studies [44], [45], curriculum learning stands out as an effective method to enhance an agent’s generalization ability and training efficiency.”
Liu, Y., Feng, X., Wang, Y., Yang, W., & Ming, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00439 · Cited by 27
“Furthermore, a curriculum learning [2] based optimization algorithm is introduced to utilize artificial images and explore the diversity of input.”
MethodCheng, X., Jin, T., Huang, R., Li, L., Wang, L., Wang, Z., Wang, Y., Liu, H., Yin, A., & Zhao, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01442 · Cited by 39
“We also propose a curriculum-learning [7] based strategy to adjust the mixing ratio as the training progresses and cross-modality integration deepens.”
MethodKwon, N., Yoo, Y., & Lee, B. (2024) · IEEE Access · doi:10.1109/access.2024.3417180 · Cited by 16
Multiple Access via Curriculum Multitask HAPPO Based on Dynamic Heterogeneous Wireless Network
Han, M., Chen, Z., & Sun, X. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3432530 · Cited by 19
“To address it, we introduce the curriculum learning (CL) in the CMHA [24].”
MethodFGNN2: A Powerful Pretraining Framework for Learning the Logic Functionality of Circuits
Wang, Z., Bai, C., He, Z., Zhang, G., Xu, Q., Ho, T.-Y., Huang, Y., & Yu, B. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3434464 · Cited by 37
“We have incorporated a curriculum learning [40] scheme into our pretraining procedure to address the challenges associated with learning difficulty.”
TrajFine: Predicted Trajectory Refinement for Pedestrian Trajectory Forecasting
Wang, K.-L., Tsao, L.-W., Wu, J.-C., Shuai, H.-H., & Cheng, W.-H. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00451 · Cited by 23
“From the data perspective, the early work [3] involves predefined difficulty levels according to samples, while Spitkovsky et al .”
MethodGSSCL: A framework for Graph Self-Supervised Curriculum Learning based on clustering label smoothing
Fu, Y.-G., Chen, X., Xu, S., Li, J., Yao, X. F., Huang, Z., & Wang, Y. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106787 · Cited by 17
Deep learning long-range information in undirected graphs with wave networks
Matlock, M. K., Datta, A., Dang, N. L., Jiang, K., & Swamidass, S. J. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852455 · Cited by 20
“The entropy of this distribution increases monotonically, in accordance with the definition of curriculum learning [51].”
MethodResultFVI: An End-to-end Vietnamese Identification Card Detection and Recognition in Images
Liem, H. D., Minh, N. Đ., Trung, N. B., Duc, H. T., Hiep, P. H., Dung, D. V., & Vu, D. H. (2018) · NAFOSTED Conference on Information and Computer Science (NICS) · doi:10.1109/nics.2018.8606831 · Cited by 19
“• We optimize the OCR module which utilizes curriculum learning schedule [1] for recognizing Vietnamese words.”
MethodOn the Use of U-Net for Dominant Melody Estimation in Polyphonic Music
Doras, G., Esling, P., & Peeters, G. G. (2019) · doi:10.1109/mmrp.2019.00020 · Cited by 15
Progressively Refined Face Detection Through Semantics-Enriched Representation Learning
Li, Z., Tang, X., Wu, X., Liu, J., & He, R. (2019) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2019.2941800 · Cited by 15
“learning [43], which is consistent with human learning style in a meaningful order.”
Background2-D Seismic Random Noise Attenuation via Self-Paced Nonnegative Dictionary Learning
Yang, Y., Gao, J., Wang, Z., et al. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2957017 · Cited by 15
“[35] proposed a machine learning algorithm based on the intrinsic learning principle of humans or animals, named as the CL.”
A Novel Cox Proportional Hazards Model for High-Dimensional Genomic Data in Cancer Prognosis
Huang, H., & Liang, Y. Q. (2019) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2019.2961667 · Cited by 17
“Recently, a novel learning strategy called self-paced learning (SPL) was proposed [19], [20].”
BackgroundTriplet-Based Wireless Channel Charting
Ferrand, P., Decurninge, A., Ordóñez, L. G., & Guillaud, M. (2020) · Globecom · doi:10.1109/globecom42002.2020.9322643 · Cited by 15
“Additionally, the selection of triplets can also be increasingly hardened so that the embedding discriminates better between the classes; such an approach is called curriculum learning or shaping [20].”
BackgroundTackling Morpion Solitaire with AlphaZero-like Ranked Reward Reinforcement Learning
Wang, H., Preuß, M., Emmerich, M., & Plaat, A. (2020) · International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC) · doi:10.1109/synasc51798.2020.00033 · Cited by 16
“AlphaGo and AlphaZero combine deep neural networks [9] and Monte Carlo Tree Search (MCTS) [10] in a self-play framework that learns by curriculum learning [11].”
BackgroundPathological Image Classification Based on Hard Example Guided CNN
Wang, Y., Peng, T., Duan, J., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3003070 · Cited by 15
“Motivated by curriculum learning [20], Kumar et al .”
MethodBIDI: A classification algorithm with instance difficulty invariance
Yu, S., Li, X., Wang, H., et al. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113920 · Cited by 15
Explore Video Clip Order With Self-Supervised and Curriculum Learning for Video Applications
Xiao, J., Li, L., Xu, D., Long, C., Shao, J., Zhang, S., Pu, S., & Zhuang, Y. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3025661 · Cited by 15
“Aiming at training complex network in deep learning, Bengio et al. [43] proposes curriculum learning.”
BackgroundA Deep Reinforcement Learning Approach for the Pursuit Evasion Game in the Presence of Obstacles
Qi, Q., Zhang, X., & Guo, X. (2020) · IEEE International Conference on Real-time Computing and Robotics (RCAR) · doi:10.1109/rcar49640.2020.9303044 · Cited by 17
“In this paper, curriculum learning [21] is integrated with deep Q-network, which is inspired by the learning mechanism of humans, that is, learning simple problems before learning more complex ones.”
BackgroundReal-Time Decision Making for a Car Manufacturing Process Using Deep Reinforcement Learning
Gros, T. P., Groß, J., & Wolf, V. (2020) · Winter Simulation Conference · doi:10.1109/wsc48552.2020.9383884 · Cited by 19
“To handle the former, we roughly follow the idea of curriculum learning (Bengio et al. 2009).”
BackgroundXAI-N: Sensor-based Robot Navigation using Expert Policies and Decision Trees
Roth, A. M., Liang, J., & Manocha, D. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636759 · Cited by 20
“[44] We follow the procedure used in [45].”
MethodHow do People Train a Machine? Strategies and (Mis)Understandings
Sanchez, T., Caramiaux, B., Françoise, J., Bevilacqua, F. P., & Mackay, W. E. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449236 · Cited by 27
“This finding is supported by the Machine Learning literature, although rather scarcely explored, which shows that simulating a curriculum improves the performance [5].”
ResultRevisiting Optical Flow Estimation in 360 Videos
Bhandari, K., Zong, Z., & Yan, Y. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412035 · Cited by 22
“Training architectures like this using better strategy to cope with gradually increasing task is an adoption of a popular philosophy, curriculum learning[15].”
Huerta, E. A., & Zhao, Z. (2021) · Springer eBooks · doi:10.1007/978-981-15-4702-7_47-1 · Cited by 24
“Thus, the methods presented in [56, 58] showed how to adapt curriculum learning [59], originally developed in the context of image processing, to do classification or detection of noisy and weak signals.”
MethodA conceptual framework for externally-influenced agents: an assisted reinforcement learning review
Bignold, A., Cruz, F., Taylor, M., Brys, T., Dazeley, R., Vamplew, P., & Foale, C. (2021) · Deakin Research Online (Deakin University) · Cited by 15
Normalized Human Pose Features for Human Action Video Alignment
Liu, J., Shi, M., Chen, Q., Fu, H., & Tai, C.-L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01132 · Cited by 22
“Adaptive mining also facilitates curriculum learning [3], where the training starts from easy negative pairs and gradually shifts to semi-hard pairs.”
BackgroundMulti-Level Curriculum for Training A Distortion-Aware Barrel Distortion Rectification Model
Liao, K., Lin, C., Liao, L., Zhao, Y., & Lin, W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00435 · Cited by 16
“Bengio et al. [4] thoroughly described the concept, details, and experiment of curriculum learning.”
BackgroundMethodInfluentialDifferentiated Learning for Multi-Modal Domain Adaptation
Lv, J., Liu, K., & He, S. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475660 · Cited by 23
“The core idea of curriculum learning [1, 16, 19] is that the model should start learning from simple samples and gradually transition to difficult samples, which make the training process more stable.”
BackgroundReview and Arrange: Curriculum Learning for Natural Language Understanding
Zhang, L., Mao, Z., Xu, B., Wang, Q., & Zhang, Y. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3121986 · Cited by 16
Matsuzaki, S., & Hasegawa, Y. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812011 · Cited by 21
Implicit Event Argument Extraction With Argument-Argument Relational Knowledge
Wei, K., Sun, X., Zhang, Z., Jin, L., Zhang, J., Jianwei, L. V., & Guo, Z. M. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3218830 · Cited by 26
“Based on this observation, inspired by the curriculum theory [23] that a machine learning model could be trained better by feeding data following the order from easiest to hardest, we introduce a curriculum learning strategy during training to promote the learning of the student model.”
BackgroundMethodRobust Video watermarking based on deep neural network and curriculum learning
Ke, Z., Huang, H., Liang, Y., Ding, Y., Cheng, X., & Wu, Q. (2022) · IEEE International Conference on e-Business Engineering (ICEBE) · doi:10.1109/icebe55470.2022.00023 · Cited by 20
Global context and boundary structure-guided network for cross-modal organ segmentation
Guo, X., Xie, H., Xu, H. Y., & Zhang, Y. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102252 · Cited by 17
LAMP: Large Deep Nets with Automated Model Parallelism for Image Segmentation
Zhu, W., Zhao, C., Li, W., Roth, H. R., Xu, Z., & Xu, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59719-1_37 · Cited by 24
“learning theory such as curriculum learning [1,12], a model can be well trained by firstly being fit easy samples/tasks and later being fit hard samples/tasks.”
BackgroundNeural Networks for Sequential Data: a Pre‐training Approach based on Hidden Markov Models
Pasa, L., Testolin, A., & Sperduti, A. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.11.081 · Cited by 16
“A final approach to pre-train sequential models, fromwhich we take inspiration in our work, is related to the idea of curriculum learning [43].”
Towards Black-box Iterative Machine TeachingPreprint
Liu, W., Dai, B., Li, X., Liu, Z., Rehg, J. M., & Song, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.07742 · Cited by 20
“Besides various applications, machine teaching also has nice connections with curriculum learning (Bengio et al., 2009; Hinton et al., 2015).”
BackgroundTunneling Neural Perception and Logic Reasoning through Abductive LearningPreprint
Dai, W.-Z., Xu, Q., Yu, Y., & Zhou, Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01173 · Cited by 23
“In fact, some sets of examples (such as the arithmetic equations “1+10=11, 11+100=111”) can achieve maximum consistency using just one bitwise addition rule my op(1,0,[1]) , which results in an incomplete ∆ tC for defining the target concept.”
BackgroundMethodInfluentialTowards Training Recurrent Neural Networks for Lifelong LearningPreprint
Sodhani, S., Chandar, S., & Bengio, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.07017 · Cited by 13
“Specifically, we train the models in the curriculum style setup (Bengio et al., 2009) where the tasks are ordered by difficulty.”
MethodSelf-citationExploring Benefits of Transfer Learning in Neural Machine TranslationPreprint
Kocmi, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01622 · Cited by 27
“Furthermore, we have tackled problems connected with catastrophic forgetting in Kocmi and Bojar (2017a) when experimenting with curriculum learning (Bengio et al., 2009).”
Background
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.