Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 1,401–1,500 of 5,240
Collaborative Learning-Based Scheduling for Kubernetes-Oriented Edge-Cloud Network
Shen, S., Han, Y., Wang, X., Wang, S., & Leung, V. C. M. (2023) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2023.3267168 · Cited by 31
“Inspired by curriculum learning proposed in [36], we first use simple and short service request sequences for training, and then moderately introduce more sophisticated request sequences step by step, so that the scheduling policy can be gradually improved.”
MethodDeep Fuzzy Variable C-Means Clustering Incorporated With Curriculum Learning
Gong, M., Zhao, Y., Li, H., Qin, A. K., Xing, L., Li, J., Liu, Y., & Liu, Y. (2023) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2023.3283046 · Cited by 25
“Curriculum learning [33], [34] is a learning regime inspired by the learning process of humans and animals that gradually incorporates samples into training set from easy to complex by assigning a weight to each training sample.”
BackgroundDual Arbitrary Scale Super-Resolution for Multi-contrast MRI
Zhang, J., Chi, Y., Lyu, J., Yang, W., & Tian, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43999-5_27 · Cited by 23
Timiryasov, I., & Tastet, J. (2023) · BabyLM Challenge · doi:10.18653/v1/2023.conll-babylm.24 · Cited by 92
Why curriculum learning & self-paced learning work in big/noisy data: A theoretical perspective
Gong, T., Zhao, Q., Meng, D., & Xu, Z. (2015) · Big Data and Information Analytics · doi:10.3934/bdia.2016.1.111 · Cited by 37
“This thus complies with the weight-increasing condition defined for a curriculum in [2].”
BackgroundInfluentialResidual neural networks for speech recognition
Vydana, H. K., & Vuppala, A. K. (2017) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2017.8081266 · Cited by 21
“Learning strategies like curriculum learning, continuation methods [9], mollifying networks [10] and use of noisy activation functions [11] have been studied to aid the optimization of a highly non convex objective functions.”
BackgroundZhu, Y., Wan, J., Zhou, Z., Chen, L., Qiu, L., Zhang, W., Jiang, X., & Yu, Y. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331232 · Cited by 34
“thm of our proposed method. Note that instead of training the judger to convergence at the beginning, the judger and the generator are trained alternately. From the perspective of curriculum learning [8], by gradually increasing the complexity of the generator’s training objective, it improves the generalization abili…”
MethodRepeat before Forgetting: Spaced Repetition for Efficient and Effective Training of Neural Networks
Amiri, H., Miller, T. A., & Savova, G. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1255 · Cited by 22
“Bengio et al. (2009) and Kumar et al. (2010) also developed cognitively-motivated training paradigms which are inspired by the principle that learning can be more effective when training starts with easier concepts and gradually proceeds with more difficult ones.”
BackgroundQi, Q., Lin, X., Chen, C., Xie, W., Huang, Y., Ding, X., Liu, X., & Yu, Y. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3021558 · Cited by 22
“Curriculum learning [42] imitates human learning and uses easy-to-hard sample selection strategies to arrange training samples.”
BackgroundEGDCL: An Adaptive Curriculum Learning Framework for Unbiased Glaucoma Diagnosis
Zhao, R., Chen, X., Chen, Z., & Li, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_12 · Cited by 26
“Curriculum learning [1] and self-paced learning [15] represents a learning regime inspired by the learning proceeds of humans that gradually proceeds from easy to more complex or hard to deal with the samples imbalance.”
BackgroundPro-UIGAN: Progressive Face Hallucination From Occluded Thumbnails
Zhang, Y., Yu, X., Lu, X., & Liu, P. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3167280 · Cited by 23
“[41], we define a reasonable multi-stage progressive hallucination strategy.”
MethodMachine Teaching for Bayesian Learners in the Exponential FamilyPreprint
Zhu, J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.4947 · Cited by 72
“The third thread is the research on better ways to training machine learners such as curriculum learning or easy-to-hard ordering of training items [3, 15, 16], and optimal reward design in reinforcement learning [21].”
BackgroundThe Intentional Unintentional Agent: Learning to Solve Many Continuous Control Tasks SimultaneouslyPreprint
Cabi, S., Colmenarejo, S. G., Hoffman, M. W., Denil, M., Wang, Z., & de Freitas, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.03300 · Cited by 33
“Our work is related to learning neural networks with a static or adaptive curricula [19, 20, 21], and to learning curricula for training neural networks using bandit techniques and Bayesian optimization [22, 23].”
MethodUnderstanding Early Word Learning in Situated Artificial AgentsPreprint
Hill, F., Clark, S. R. L., Hermann, K. M., & Blunsom, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.09867 · Cited by 38
“Deep learning researchers have also reported faster or more effective learning when training examples are ordered by some metric of complexity, an approach referred to as curriculum learning (Bengio et al., 2009).”
BackgroundTowards Practical Multi-Object Manipulation using Relational Reinforcement LearningPreprint
Li, R., Jabri, A., Darrell, T. J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11032 · Cited by 21
“Prior work has shown that ordering tasks by heuristic measures of difficulty can be effective [45], [46].”
BackgroundReplay in Deep Learning: Current Approaches and Missing Biological ElementsPreprint
Hayes, T. L., Krishnan, G. P., Bazhenov, M., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.04132 · Cited by 21
“More interestingly, hindsight experience replay can serve as a form of curriculum learning (Bengio, Louradour, Collobert, & Weston, 2009) by structuring the rewards such that they start off simple and grow increasingly more complex during training.”
ST++: Make Self-training Work Better for Semi-supervised Semantic SegmentationPreprint
Yang, L., Zhuo, W., Qi, L., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05095 · Cited by 21
“Moreover, both aforementioned works adopt the plain training pipeline, whereas we further propose the ST++ to safely exploit unlabeled images in a curriculum learning manner [5].”
BackgroundReplay-Guided Adversarial Environment Design
Jiang, M., Dennis, M., Parker-Holder, J., Foerster, J., Grefenstette, E., & Rocktäschel, T. (2021) · arXiv · doi:10.48550/arxiv.2110.02439 · Cited by 21
“In inducing parallel curricula, DCD follows a rich lineage of curriculum learning methods [2, 32, 28, 23].”
BackgroundLearning to navigate cloth using haptics
Clegg, A., Yu, W., Erickson, Z., Tan, J., Liu, C. K., & Turk, G. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206110 · Cited by 20
“The neural network is trained using Trust Region Policy Optimization (TRPO) [22] with a curriculum learning strategy [31].”
MethodA Deep Model for Partial Multi-label Image Classification with Curriculum-based Disambiguation
Sun, F. J., Xie, M.-K., & Huang, S.-J. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1439-3 · Cited by 27
“Early methods focus on designing easy-to-hard curriculums based on prior knowledge [14].”
BackgroundTeng, X., Zhang, L., Gao, P., Yu, C., & Sun, S. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112627 · Cited by 27
Self-learning for face clustering
Shi, X., Guo, Z., Xing, F., Cai, J., & Yang, L. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.02.008 · Cited by 20
Does an LSTM forget more than a CNN? An empirical study of catastrophic forgetting in NLP
Arora, G. S., Rahimi, A., & Baldwin, T. (2019) · Cited by 35
Greedy auto-augmentation for n-shot learning using deep neural networks
Naghizadeh, A., Metaxas, D., & Liu, D. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.11.015 · Cited by 26
Kontes, G. D., Scherer, D. D., Nisslbeck, T. N., Fischer, J., & Mutschler, C. (2020) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc45102.2020.9294396 · Cited by 27
“Note here that our initial reward design involved a Curriculum Learning setup [29]: once the agent learned to overtake the vehicle, we would add another term in the reward function penalizing collisions and continue training.”
MethodFlexible lane detection using CNNs
Li, H., & Xizhou, L. (2021) · International Conference on Computer Technology and Media Convergence Design (CTMCD) · doi:10.1109/ctmcd53128.2021.00057 · Cited by 20
Deep Reinforcement Learning of Map-Based Obstacle Avoidance for Mobile Robot Navigation
Chen, G., Pan, L., Chen, Y., Xu, P., Wang, Z., Wu, P., Ji, J., & Chen, X. (2021) · SN Computer Science · doi:10.1007/s42979-021-00817-z · Cited by 27
“Curriculum learning [32] aims to improve learning performance by designing appropriate curriculums for progressive learners from simple to difficult.”
BackgroundSrinidhi, C. L., & Martel, A. L. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00069 · Cited by 32
“In relation to the CL paradigm introduced in [6, 19, 38], we first start by asking two very fundamental questions: (i) how to determine the notion of example difficulty (i.”
BackgroundCognitive multi-modal consistent hashing with flexible semantic transformation
An, J., Luo, H., Zhang, Z., et al. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102743 · Cited by 20
Simple Baseline for Single Human Motion Forecasting
Wang, C., Wang, Y., Huang, Z., & Chen, Z. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00255 · Cited by 35
Active Perception for Visual-Language Navigation
Wang, H., Wang, W., Liang, W., Hoi, S. C. H., Shen, J., & Gool, L. V. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01721-6 · Cited by 32
“To tackle these challenges, we organize the training protocol as an incremental learning procedure, inspired by the curriculum learning strategy Bengio et al. (2009).”
MethodSocial Image–Text Sentiment Classification With Cross-Modal Consistency and Knowledge Distillation
Liu, H., Li, K., Fan, J., Yan, C., Qin, T., & Zheng, Q. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3220762 · Cited by 24
“Curriculum learning [45] applies the same principle used by humans when learning knowledge, and accordingly advocates for a learning process that moves from easier samples to more difficult samples.”
BackgroundChen, X., Qi, Y., Yin, Y., et al. (2023) · Applied Sciences · doi:10.3390/app13042244 · Cited by 20
“Inspired by curriculum learning [44] in machine learning, we designed the experiment as a multi-stage training process for the path planning problem.”
Learning Needle Pick-and-Place Without Expert Demonstrations
Bendikas, R., Modugno, V., Kanoulas, D., Vasconcelos, F., & Stoyanov, D. V. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3266720 · Cited by 33
“simpler behaviors before moving on to harder ones [26].”
BackgroundFeyisa, D. W., Ayano, Y. M., Debelee, T. G., & Schwenker, F. (2023) · Sensors · doi:10.3390/s23156781 · Cited by 30
“This principle is called curriculum learning [111].”
Make Segment Anything Model Perfect on Shadow Detection
Chen, X.-D., Wu, W., Yang, W., Qin, H., Wu, X.-T., & Mao, X. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3332257 · Cited by 37
“This characteristic can help us to divide the training data into easy-to-hard subsets, facilitating curriculum learning [47].”
BackgroundControllable augmentations for video representation learning
Qian, R., Lin, W., See, J., & Li, D. (2024) · Visual Intelligence · doi:10.1007/s44267-023-00034-7 · Cited by 27
“[60, 61], it is promising to learn from easier samples and then gradually expand to more dif-ficult tasks.”
iPINNs: incremental learning for Physics-informed neural networks
Dekhovich, A., Sluiter, M. H. F., Tax, D. M. J., & Bessa, M. A. (2024) · Engineering With Computers · doi:10.1007/s00366-024-02010-1 · Cited by 31
“Analogously, curriculum regularization (similar to curriculum learning [68]) is proposed in [35] to find good initial weights.”
Meta-learning in Healthcare: A Survey
Rafiei, A., Moore, R., Jahromi, S., Hajati, F., & Kamaleswaran, R. (2024) · SN Computer Science · doi:10.1007/s42979-024-03166-9 · Cited by 42
“CL-selector is a training strategy to afford an easy-to-hard learning scheme for MAML [79].”
Development of Deep-Learning-Based Autonomous Agents for Low-Speed Maneuvering in Unity
Berta, R., Lazzaroni, L., Capello, A., Cossu, M., Forneris, L., Pighetti, A., & Bellotti, F. (2024) · Journal of Intelligent and Connected Vehicles · doi:10.26599/jicv.2023.9210039 · Cited by 27
A Gradient-based Adaptive Learning Framework for Efficient Personal Recommendation
Ning, Y., Shi, Y., Hong, L., Rangwala, H., & Ramakrishnan, N. (2017) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3109859.3109909 · Cited by 23
“Personalization/Multi-Task Learning: e importance of a personalized application extends to many elds such as social networks [20, 22] and ranking system [29] while social network has also been incorporated in personalization [9].”
BackgroundInfluentialGeneralised Discriminative Transform via Curriculum Learning for Speaker Recognition
Marchi, E., Shum, S. H., Hwang, K., et al. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461296 · Cited by 20
“We then propose an LSTM model together with two learning strategies aimed at improving generalisation: curriculum learning [9] and multi-style training.”
MethodAn Optimization Approach to Improve Classification Performance in Cancer and Diabetes Prediction
Helal, M. A., Chowdhury, A. I., Islam, A., Ahmed, E., Mahmud, M. S., & Hossain, S. (2019) · International Conference on Electrical, Computer and Communication Engineering · doi:10.1109/ecace.2019.8679413 · Cited by 21
“It has already been proved that curriculum learning performs better in some particular problems of machine learning [6].”
BackgroundMethodFCSR-GAN: End-to-end Learning for Joint Face Completion and Super-resolution
Cai, J., Han, H., Shan, S., & Chen, X. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756607 · Cited by 20
“In the first stage training, we adopt a curriculum strategy [23] to gradually increase the difficulty of network training.”
MethodNeural Task Planning With AND–OR Graph Representations
Chen, T., Chen, R., Nie, L., Luo, X., Liu, X., & Lin, L. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2870062 · Cited by 19
“over, because the automatically augmented set includes some difficult samples with uncertain and even incorrect labels, we further incorporate curriculum learning [21], [22] to address this”
BackgroundMethodContext-Dependent Translation Selection Using Convolutional Neural Network
Hu, B., Tu, Z., Lu, Z., Li, H., & Chen, Q. (2015) · doi:10.3115/v1/p15-2088 · Cited by 26
“It consists of two components: • convolutional sentence model that summarizes the meaning of the source sentence and the target phrase; • matching model that compares the two representations with a multi-layer perceptron (Bengio, 2009).”
BackgroundAdaptive Curriculum Generation from Demonstrations for Sim-to-Real Visuomotor Control
Hermann, L., Argus, M. J., Eitel, A., Amiranashvili, A., Burgard, W., & Brox, T. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197108 · Cited by 32
“[8] shows that curriculum learning provides benefits for classical supervised learning problems such as shape classification.”
BackgroundBézier Curve Based Continuous and Smooth Motion Planning for Self-Learning Industrial Robots
Scheiderer, C., Thun, T., & Meisen, T. (2019) · Procedia Manufacturing · doi:10.1016/j.promfg.2020.01.054 · Cited by 22
“We will further explore the possibility of curriculum learning [20] for a robust step-by-step increase of the degree of Bezier curves.”
BackgroundLearning From Web Data With Self-Organizing Memory Module
Tu, Y., Niu, L., Chen, J., Cheng, D., & Zhang, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01286 · Cited by 31
Occluded Facial Expression Recognition with Step-Wise Assistance from Unpaired Non-Occluded Images
Xia, B., & Wang, S. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413773 · Cited by 23
Neural Knapsack: A Neural Network Based Solver for the Knapsack Problem
Nomer, H. A. A., Alnowibet, K. A., Elsayed, A., & Mohamed, A. W. (2020) · IEEE Access · doi:10.1109/access.2020.3044005 · Cited by 27
“All the models, unless otherwise mentioned, was trained using naive curriculum learning strategy, more details on curriculum learning strategy can be found in [56].”
MethodA 2021 update on cancer image analytics with deep learning
Kurian, N. C., Sethi, A., Konduru, A. R., Mahajan, A., & Rane, S. U. (2021) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1410 · Cited by 24
“For instance, use of curriculum learning (Bengio, Louradour, Collobert, & Weston, 2009), self-paced learning (Kumar, Packer, & Koller, 2010), or other sample reweighting schemes (Ren, Zeng, Yang, & Urtasun, 2018) has been proposed, where the individual training examples are differentially weighted…”
MethodTraining Deep Networks from Zero to Hero: avoiding pitfalls and going beyond
Ponti, M. A., dos Santos, F. P., Ribeiro, L. S. F., & Cavallari, G. B. (2021) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi54419.2021.00011 · Cited by 32
Improving Deep Metric Learning by Divide and Conquer
Sanakoyeu, A., Ma, P., Tschernezki, V., & Ommer, B. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3113270 · Cited by 24
“But many research works [76], [79], [80], [81], [82] argue that a smooth transition from an easy training setup to a more complex one is usually a more fruitful strategy which improves training convergence and generalization.”
BackgroundAn efficient curriculum learning-based strategy for molecular graph learning
Gu, Y., Zheng, S., Xu, Z., Yin, Q., Li, L., & Li, J. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac099 · Cited by 22
Huang, R., & Wong, V. W. S. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3159187 · Cited by 20
“Curriculum learning [26] is a technique for modifying the system transition history observed by the learning agent such”
BackgroundMethodOne Step Back, Two Steps Forward: Interference and Learning in Recurrent Neural Networks
Beer, C., & Barak, O. (2019) · Neural Computation · doi:10.1162/neco_a_01222 · Cited by 26
“Shaping, or choosing an optimal sequence of training examples, is a topic of importance in both animal training and machine learning research [29,30].”
BackgroundEvolving Culture vs Local MinimaPreprint
Bengio, Y. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1203.2990 · Cited by 25
“Deep architectures (Bengio, 2009) are parametrized families of functions which can be used to model data using multiple levels of representation.”
BackgroundMethodResultInfluentialSelf-citationCombining Strategic Learning and Tactical Search in Real-Time Strategy GamesPreprint
Barriga, N. A., Stănescu, M., & Buro, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.03480 · Cited by 20
“There are several strategies available that could help overcome these issues, such as curriculum learning (Bengio et al. 2009), reward shaping (Devlin, Kudenko, and Grze ´ s 2011), or implementing double DQN learning (Hasselt, Guez, and Silver 2016).”
BackgroundCurriculumNet: Weakly Supervised Learning from Large-Scale Web ImagesPreprint
Guo, S., Huang, W., Zhang, H., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.01097 · Cited by 20
“Curriculum learning was originally proposed in [1].”
Counterfactual Multi-Agent Reinforcement Learning with Graph Convolution CommunicationPreprint
Su, J., Adams, S., & Beling, P. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00470 · Cited by 33
“We utilized curriculum learning [1] to train our agents in total 2 million training steps.”
Huizinga, J., & Clune, J. (2022) · Evolutionary Computation · doi:10.1162/evco_a_00301 · Cited by 20
“…of tasks have been successfully applied in animal training (Skinner, 1958; Peterson, 2004), gradient-descent based machine learning (Elman, 1993; Bengio et al., 2009), and evolutionary algorithms (Gomez and Miikkulainen, 1997; Lewis et al., 1992, 1993; Harvey et al., 1994; Larsen and Hansen,…”
MethodLin, Y., Song, W., Li, B., Li, Y., Ni, J., Chen, H., & Li, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73016-0_7 · Cited by 86
Learning of semi-empirical neural network model of aircraft three-axis rotational motion
Вячеславович, Е. М., & Tiumentsev, Y. V. (2015) · Optical Memory and Neural Networks · doi:10.3103/s1060992x15030042 · Cited by 19
Global optimization issues in deep network regression: an overview
Palagi, L. (2018) · Journal of Global Optimization · doi:10.1007/s10898-018-0701-7 · Cited by 22
“In the same framework of continuation methods can be placed the curriculum learning proposed in [12], mainly used in the field of natural language recognition and computer vision.”
BackgroundOuyang, D., Shao, J., Zhang, Y., et al. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240622 · Cited by 19
“Inspired by the human/animal learning process, Bengio et al. [2] proposed the concept of curriculum learning, in which a model used sample data from simple to complex and is gradually trained.”
MethodAutomated curriculum generation through setter-solver interactions
Lampinen, A. K., Racanière, S., Santoro, A., Reichert, D., Firoiu, V., & Lillicrap, T. (2020) · International Conference on Learning Representations · Cited by 45
End-to-End Spoken Language Understanding Without Full Transcripts
Kuo, H.-K. J., Tüske, Z., Thomas, S., Huang, Y., Audhkhasi, K., Kingsbury, B. E. D., Kurata, G., Kons, Z., Hoory, R., & Lastras, L. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2924 · Cited by 25
“This step is a form of curriculum learning [23, 25] that gradually modifies an off-the-shelf ASR model into a full fledged SLU model.”
BackgroundMultimodal super-resolved q-space deep learning
Yu, Q., Li, Y., Zhuo, Z., Liu, Z., Liu, Y., & Ye, C. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102085 · Cited by 28
Fail-Safe Human Detection for Drones Using a Multi-Modal Curriculum Learning Approach
Safa, A., Verbelen, T., Ocket, I., Bourdoux, A., Catthoor, F., & Gielen, G. E. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3125450 · Cited by 23
“Then, the sequence Qλ is defined as a curriculum if [26]:”
BackgroundMethodInfluentialNovelD: A Simple yet Effective Exploration Criterion
Zhang, T., Xu, H., Wang, X., Wu, Y., Keutzer, K., Gonzalez, J. E., & Tian, Y. (2021) · neural information processing systems · Cited by 89
Progressive Feature Learning for Facade Parsing With Occlusions
Ma, W., Xu, S., Ma, W., Zhang, X., & Zha, H. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3152004 · Cited by 18
Disentangling Normal Aging From Severity of Disease via Weak Supervision on Longitudinal MRI
Ouyang, J., Zhao, Q., Adeli, E., Zaharchuk, G., & Pohl, K. M. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3166131 · Cited by 30
“by the idea of curriculum learning [40], [41], we designed a coarse-to-fine step-wise training strategy that gradually added”
MethodCamel: Managing Data for Efficient Stream Learning
Li, Y., Shen, Y., & Chen, L. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517836 · Cited by 38
“Curriculum learning [6] proposes to provide the network with examples of increasing difficulty.”
BackgroundLiu, W., Liu, J., Luo, Z., Zhang, H., Gao, K., & Li, J. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102931 · Cited by 27
A Relevant and Diverse Retrieval-enhanced Data Augmentation Framework for Sequential Recommendation
Bian, S., Zhao, W. X., Wang, J., & Wen, J.-R. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557071 · Cited by 31
Kaur, N., & Mittal, A. (2022) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-022-04454-z · Cited by 21
Saffari, M., Khodayar, M., & Jalali, S. M. J. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3234548 · Cited by 20
“Kothandaraman et al. [16] developed a road segmentation approach under severe weather conditions(i.e.,snow,rain,andfog)byconstructingapre-trained modelontheclearweathersamplesandtakingadvantageof curriculumlearning[17]tobridgethesourceandtargetdomains’ gapprogressively.”
BackgroundA Primer on Seq2Seq Models for Generative Chatbots
Scotti, V., Sbattella, L., & Tedesco, R. (2023) · ACM Computing Surveys · doi:10.1145/3604281 · Cited by 37
“The training process can be carried out in multiple steps, from pre-training [134, 216] to ine-tuning [210], following a so-called curriculum learning approach [15].”
BackgroundModular Reinforcement Learning for Autonomous UAV Flight Control
Choi, J., Kim, H. M., Hwang, H. J., Kim, Y.-D., & Kim, C. O. (2023) · Drones · doi:10.3390/drones7070418 · Cited by 22
ASP: Learn a Universal Neural Solver!
Wang, C., Yu, Z., McAleer, S., Yu, T., & Yang, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3352096 · Cited by 53
“Curriculum Learning - [32] first propose the concept of curriculum learning (CL) where a machine learning algorithm shouldbetrainedfromeasytaskstohardtasks.”
BackgroundWang, J., Lin, C., Nie, L., Huang, S., Zhao, Y., Pan, X., & Ai, R. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611100 · Cited by 54
“This allows the model to obtain better-generalizable local optima for pre-training on more complex scenarios [2].”
MethodImproved self-paced learning framework for nonnegative matrix factorization
Zhu, X., & Zhang, Z. (2017) · Pattern Recognition Letters · doi:10.1016/j.patrec.2017.06.016 · Cited by 21
“[26] proposed a new machine learning framework called curriculum learning (CL).”
BackgroundDiverse Ensemble Evolution: Curriculum Data-Model Marriage
Zhou, T., Wang, S., & Bilmes, J. (2018) · neural information processing systems · Cited by 32
Self-Paced Adversarial Training for Multimodal Few-Shot Learning
Pahde, F., Ostapenko, O., Jähnichen, P., Klein, T., & Nabi, M. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00029 · Cited by 25
“[1] first proposed a general learning strategy: curriculum learning.”
BackgroundIncremental Lifelong Deep Learning for Autonomous Vehicles
Pierre, J. M. (2018) · International Conference on Intelligent Transportation Systems · doi:10.1109/itsc.2018.8569992 · Cited by 19
“Curriculum learning strategies [11], where training examples are presented in a non-random order starting with the simpler/easier cases and then gradually introducing more complex/harder examples, can be helpful for finding better minima of a non-convex objective function, leading to improved generalization and faster…”
BackgroundKabir, A., Ruiz, C., & Álvarez, S. A. (2018) · Proceedings · doi:10.1109/icdm.2018.00137 · Cited by 24
“In [10], instance hardness is applied to filtering and curriculum learning [11], and their performances are compared to boosting for supervised classification.”
MethodGarbacea, C., Carton, S., Yan, S., & Mei, Q. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1409 · Cited by 18
Self-Paced Robust Learning for Leveraging Clean Labels in Noisy Data
Zhang, X., Wu, X. W., Chen, F., et al. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6166 · Cited by 19
“Inspired by the learning process of humans and animals (Bengio et al. 2009 (Meng, Zhao, and Jiang 2015) provides a theoretical analysis of the robustness of SPL , revealing that the implicit objective function of SPL has a similar configu-ration to a non-convex regularized penalty.”
MethodSelf-supervised representation learning by predicting visual permutations
Zhao, Q., & Dong, J. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106534 · Cited by 19
Flocking Control of UAV Swarms with Deep Reinforcement Leaming Approach
Peng, Y., Bai, C., Zheng, H., & Guo, J. (2020) · IEEE International Conference on Unmanned Systems (ICUS) · doi:10.1109/icus50048.2020.9274899 · Cited by 19
Coming to Grips with Age Prediction on Imbalanced Multimodal Community Question Answering Data
Figueroa, A., Peralta, B., & Nicolis, O. (2021) · Information · doi:10.3390/info12020048 · Cited by 20
“In curriculum learning [38], a learning plan is devised by ranking samples based on carefully chosen and thoughtfully organized difficulty metrics.”
Style Curriculum Learning for Robust Medical Image Segmentation
Liu, Z., Manh, V., Yang, X. S., Huang, X., Lekadir, K., Campello, V. M., Ravikumar, N., Frangi, A. F., & Ni, D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87193-2_43 · Cited by 22
“Curriculum learning methods [1,6,20] can guide the model to learn better by organising the training samples in a meaningful order.”
BackgroundAction2video: Generating Videos of Human 3D Actions
Guo, C., Zuo, X., Wang, S., Liu, X., Zou, S., Gong, M., & Cheng, L. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-021-01550-z · Cited by 34
Handwritten Information Extraction from Historical Census Documents
Nion, T., Farès Menasri, A., Louradour, J., Sibade, C., Retornaz, T., Metaireau, P.-Y., & Kermorvant, C. (2013) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2013.168 · Cited by 19
“has already been recommended for SGD under the name "curriculum" [12].”
MethodSelf-citationWatch What You Just Said: Image Captioning with Text-Conditional AttentionPreprint
Zhou, L., Xu, C., Koch, P. A., & Corso, J. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04621 · Cited by 36
“Inspired by curriculum learning [21], we train the model in three steps: 1) end-to-end gLSTM model training without CNN fine-tuning for 70,000 iterations, 2) continue training the end-to-end gLSTM model for 200,000 iterations, but with CNN fine-tuning, 3) introduce and initialize the text-conditional matrix with ones,…”
MethodSequence Transfer Learning for Neural DecodingPreprint
Elango, V., Patel, A. N., Miller, K. J., & Gilja, V. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/210732 · Cited by 22
“Curriculum learning, originally proposed by Bengio et al. (2009), increases the difficulty of the training samples across epochs and Zaremba & Sutskever (2014) proposed adding a probability for selecting the full-sequence or the curriculum-sequence.”
BackgroundTransferring Cross-domain Knowledge for Video Sign Language RecognitionPreprint
Li, D., Yu, X. F., Xu, C., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.03703 · Cited by 19
“In particular, [21] combines curriculum learning [1] and selfpace learning [19] to learn a concept detector.”
BackgroundGlancing Transformer for Non-Autoregressive Neural Machine TranslationPreprint
Qian, L., Zhou, H., Bao, Y., Wang, M., Qiu, L., Zhang, W., Yu, Y. H., & Li, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.07905 · Cited by 18
“With a similar idea with curriculum learning (Bengio et al., 2009), we propose a curriculum sampling strategy to decide the number that aims at reducing more difficulty from the start and gradually approaching the simultaneous generation setting as the model is trained better.”
BackgroundMethodSim-to-real: Six-legged Robot Control with Deep Reinforcement Learning and Curriculum Learning
Qin, B., Gao, Y., & Bai, Y. (2019) · 2019 4th International Conference on Robotics and Automation Engineering (ICRAE) · doi:10.1109/icrae48301.2019.9043822 · Cited by 18
“The curriculum learning was firstly proposed by Bengio in 2009[13].”
Background
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.