Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 1,901–2,000 of 5,240
Zhao, Y., & Lu, H. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.12.115 · Cited by 12
Using Deep Convolutional LSTM Networks for Learning Spatiotemporal Features
Courtney, L., & Sreenivas, R. S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-41299-9_24 · Cited by 18
Khoshkholgh, M. G., & Yanıkömeroğlu, H. (2020) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2020.3045005 · Cited by 13
“We should emphasize that our approach is different than imitation learning [15], learning from demonstration [16], and curriculum learning [17], whereby the main focus is to reuse/transfer the knowledge gathered from another setting often involving easier tasks.”
BackgroundGao, Y., & Wu, L. (2021) · Electronics · doi:10.3390/electronics10131533 · Cited by 13
“Furthermore, in the primary stage of training, inspired by curriculum learning [17], we designed a method called network curriculum learning to speed up the training progress.”
Knowledge-Guided Multiview Deep Curriculum Learning for Elbow Fracture Classification
Luo, J., Kitamura, G., Arefan, D., Doğanay, E., Panigrahy, A., & Wu, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87589-3_57 · Cited by 14
“It was first introduced by Bengio et al. in [1] to enable the machine learning to mimic human learning by training a machine learning model first with ”easier” samples and then transition to ”harder” samples.”
Negative-ResNet: noisy ambulatory electrocardiogram signal classification scheme
Chen, Z., Lin, Z., Wang, P., & Ding, M. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05635-7 · Cited by 23
“Other approaches including dataset pruning [37]–[39], sample choosing [40]–[43], sample importance weighting [44]– [47] also show significant improvement on noisy data classification.”
BackgroundZhang, D., Li, Y., Zhou, Q. S., Ma, S., Li, Y., Cao, Y., & Zheng, H.-T. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095675 · Cited by 14
“The idea of Curriculum Learning is to train models from easy to hard, which is proposed in (Bengio et al., 2009).”
BackgroundLearning Efficiently in Semantic Based Regularization
Diligenti, M., Gori, M., & Scoca, V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46227-1_3 · Cited by 10
A Framework for Searching for General Artificial IntelligencePreprint
Rosa, M., Feyereisl, J., & Collective, T. G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00685 · Cited by 43
“Gradual learning has a number of other benefits [Gulcehre et al., 2016, Gulcehre and Bengio, 2016, Bengio et al., 2009]18 over standard18 Sinno J Pan and Qiang Yang.”
BackgroundInfluentialFew-shot Object DetectionPreprint
Dong, X., Zheng, L., Ma, F., Yang, Y., & Meng, D. (2017) · arXiv (Cornell University) · Cited by 14
Connecting the Dots Between MLE and RL for Sequence PredictionPreprint
Tan, B., Hu, Z., Yang, Z., Salakhutdinov, R., & Xing, E. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.09740 · Cited by 20
“The easy-to-hard learning paradigm resembles the curriculum learning (Bengio et al., 2009).”
BackgroundCurriculum Model Adaptation with Synthetic and Real Data for Semantic Foggy Scene UnderstandingPreprint
Dai, D., Sakaridis, C., Hecker, S., & Gool, L. V. (2019) · arXiv (Cornell University) · doi:10.3929/ethz-b-000342096 · Cited by 12
Semi-Supervised Learning for Cancer Detection of Lymph Node MetastasesPreprint
Jaiswal, A. K., Panshin, I., Shulkin, D., Aneja, N., & Abramov, S. (2019) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.5281/zenodo.3251086 · Cited by 41
On Multi-Agent Learning in Team Sports GamesPreprint
Zhao, Y., Borovikov, I., Rupert, J., Somers, C., & Beirami, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.10124 · Cited by 18
“To achieve faster convergence, we rely on curriculum learning (Bengio et al., 2009) in our training.”
MethodYin, X., & May, J. (2019) · arXiv (Cornell University) · Cited by 11
Ecological Reinforcement LearningPreprint
Co-Reyes, J. D., Sanjeev, S., Berseth, G., Gupta, A., & Levine, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12478 · Cited by 29
“In practice, a variety of heuristics have been used to simplify RL problems, including reward shaping [33, 13, 9] and manual or automated design of curricula [21, 7, 20, 37, 50, 23, 2, 43, 34, 18, 17, 38, 49].”
MethodTang, G., Kumar, N., & Michmizos, K. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.01157 · Cited by 12
“This form of curriculum training has been shown to result in better generalization and faster convergence [19], [20].”
BackgroundCDL: Curriculum Dual Learning for Emotion-Controllable Response GenerationPreprint
Shen, L., & Feng, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00329 · Cited by 12
“559 (Bengio et al., 2009) is to design an evaluation for”
BackgroundAn Embarrassingly Simple Approach for Trojan Attack in Deep Neural NetworksPreprint
Tang, R., Du, M., Liu, N., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08131 · Cited by 12
“1) Inspired by the curriculum learning[5], which gradually increases the complexity of inputs to benefit model training.”
BackgroundLearning to Teach with Deep Interactions.Preprint
Yang, F., Xia, Y., Wu, L., Xie, S., Liu, W., Bian, J., Qin, T., Li, X., & Liu, T. (2020) · arXiv (Cornell University) · Cited by 11
Critical Thinking for Language ModelsPreprint
Betz, G., Voigt, C., & Richardson, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.07185 · Cited by 24
“• In addition, the idea of curriculum learning for ML (Bengio et al., 2009) might be given a try.”
BackgroundUnderstanding the role of importance weighting for deep learningPreprint
Xu, D., Ye, Y., & Ruan, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.15209 · Cited by 43
“Examples include the curriculum learning (Bengio et al., 2009), mentor net (Jiang et al., 2018), co-teaching (Han et al., 2018) and knowledge distillation (Li et al., 2017; Hinton et al., 2015), where auxiliary models are employed (replacing the oracle) to represent the hardness of each data point.”
BackgroundMethodInfluentialMulti-stage Pre-training over Simplified Multimodal Pre-training Models
Liu, T., Feng, F., & Wang, X. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.199 · Cited by 11
“Existing research on curriculum learning (Bengio et al., 2009) has shown that imitating the process of human learning by gradually increasing the difficulty of a task from simple to complex in stages helps to make better use of different types of data and effectively improve the performance of learning.”
From Detection to Understanding: A Systematic Survey of Deep Learning for Scene Text Processing
Liu, Z., Song, R., Li, K., & Li, Y. (2025) · Applied Sciences · doi:10.3390/app15179247 · Cited by 18
Sun, L., Zhao, Z., Zhao, X., et al. (2026) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2026.3672419 · Cited by 11
Learning Words by Drawing Images
Surís, D., Recasens, A., Bau, D., Harwath, D. F., Glass, J. R., & Torralba, A. B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00213 · Cited by 12
“Curriculum learning: The proposal that a training curriculum should be tailored to the evolving needs of the learner is foundational in machine learning [16, 10].”
BackgroundBandits Don’t Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits
Kreutzer, J., Vilar, D., & Sokolov, A. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.274 · Cited by 25
“Such approaches are brittle in that they may not generalize well across tasks, and there has been evidence that even the reverse of the initially hypothesized order works well (Bengio et al., 2009; Wang et al., 2018; Zhang et al., 2018).”
Qing, Z., Zhang, S., Huang, Z., Xu, Y., Wang, X., Tang, M., Gao, C., Jin, R., & Sang, N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01345 · Cited by 26
“Curriculum learning [4] shows that models can learn much better when the training examples are not randomly provided but organized in a meaningful order, from easy examples to the hard ones.”
BackgroundA fundus image classification framework for learning with noisy labels
Hu, T., Yang, B., Guo, J., Zhang, W., Liu, H., Wang, N., & Li, H. (2023) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2023.102278 · Cited by 13
Data Selection Curriculum for Neural Machine Translation
Mohiuddin, T., Koehn, P., Chaudhary, V., Cross, J., Bhosale, S., & Joty, S. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.113 · Cited by 33
Investigating the Sample Weighting Mechanism Using an Interpretable Weighting Framework
Zhou, X., Wu, O., & Li, M. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3316168 · Cited by 17
“Bycontrast,theweightingstrategiesinCurriculumLearning (CL)[3]andSelf-pacedLearning(SPL)[4]sethighweightson easilyclassifiedsamples.”
BackgroundInfluentialNi, W., Liu, J., Li, Z., Liu, P., & Liang, H. (2023) · Mathematics · doi:10.3390/math11194211 · Cited by 11
Nayak, R., Pati, U. C., & Das, S. K. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17889-z · Cited by 12
Automated Parking in CARLA: A Deep Reinforcement Learning-Based Approach
Lazzaroni, L., Pighetti, A., Bellotti, F., et al. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-3-031-48121-5_50 · Cited by 11
Zheng, S., Li, Z., Li, M., & Ke, Z. (2024) · IET Intelligent Transport Systems · doi:10.1049/itr2.12494 · Cited by 13
Redirection Strategy Switching: Selective Redirection Controller for Dynamic Environment Adaptation
Lee, H. J., Jeon, S.-B., Cho, Y.-H., & Lee, I. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3372056 · Cited by 10
“This challenge can be addressed with curriculum learning [5], which can be employed to train SRC in a variety of environments.”
BackgroundA Category-Aware Curriculum Learning for Data-Free Knowledge Distillation
Li, X., Jiao, L., Sun, Q., Liu, F., Liu, X., Li, L., Chen, P., & Yang, S. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3395844 · Cited by 12
“Curriculum learning imitates the learning process of human education, and aims to make the model realize progressive learning from easy to difficult [32], [36].”
BackgroundMethodShi, J., Zhang, T., Zong, Z., Chen, S., Xin, J., & Zheng, N. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3448237 · Cited by 15
“Inspired by educational curricula for humans, curriculum learning is a methodology where a machine learning model is initially trained on simpler data, progressively moving to more complex datasets [9].”
MethodChen, C., Song, T., Mo, L., Lv, M., & Lin, D. (2025) · Aerospace · doi:10.3390/aerospace12030265 · Cited by 11
“The phased opponent policy pool introduces the curriculum learning [41] technique to provide easy-to-hard opponents for RL training, improving the training efficiency.”
CTNet: A Convolutional Transformer Network for Color Image Steganalysis
Wei, K., Luo, W., Tan, S., & Huang, J. (2025) · Journal of Computer Science and Technology · doi:10.1007/s11390-023-3006-3 · Cited by 12
GIT-CXR: End-to-End Transformer for Chest X-Ray Report Generation
Sîrbu, I., Sîrbu, I.-R., Bogojeska, J., & Rebedea, T. E. (2025) · Information · doi:10.3390/info16070524 · Cited by 16
Neural Event Extraction from Movies Description
Tozzo, A., Jovanovic, D., & Amer, M. (2018) · Workshop on Storytelling (StoryNLP) · doi:10.18653/v1/w18-1507 · Cited by 11
“Prior work in curriculum learning ((Bengio et al., 2009)) explored shape recognition and language modeling.”
BackgroundObject detection based on semi-supervised domain adaptation for imbalanced domain resources
Li, W., Wang, M., Wang, H., & Zhang, Y. (2020) · Machine Vision and Applications · doi:10.1007/s00138-020-01068-3 · Cited by 11
“[54] propose the curriculum learning (CL) paradigm, which implies that a self-training procedure should be performed by gradually feeding samples fromeasy to hard.”
BackgroundLooc: Localize Overlapping Objects with Count Supervision
Laradji, I., Pardinas, R., Rodríguez, P., & Vázquez, D. (2020) · Proceedings - International Conference on Image Processing · doi:10.1109/icip40778.2020.9191122 · Cited by 10
“The procedure ends when ri equals ci for all images, which closely resembles curriculum learning [3] under an expectation maximization framework [1].”
MethodUnderwater Object Detection using Transfer Learning with Deep Learning
Kaiyan, Z., Li, X., & Weibo, S. (2020) · International Conference on Computers, Information Processing and Advanced Education (CIPAE) · doi:10.1145/3419635.3419678 · Cited by 11
“For SSD, we use VGG16 [17] as the backbone network and conduct experiments on SSD with input sizes, SSD512.”
MethodIntrinsically Motivated Exploration of Learned Goal Spaces
Laversanne-Finot, A., Péré, A., & Oudeyer, P. (2021) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2020.555271 · Cited by 17
“For example, it can be quite difficult to specify the task in terms of reward (Christiano et al., 2017) or learning the task may require mastering many intermediate steps (Bengio et al., 2009).”
BackgroundA Curriculum Learning Approach for Pain Intensity Recognition from Facial Expressions
Mallol-Ragolta, A., Liu, S., Cummins, N., & Schuller, B. W. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00083 · Cited by 13
“2) Curriculum Learning Approach: This approach uses a CL methodology [4], which is based on progressively adding complexity to the system to benefit the generalisation capabilities of the trained models.”
TraND: Transferable Neighborhood Discovery for Unsupervised Cross-Domain Gait Recognition
Zheng, J., Liu, X., Yan, C. C., Zhang, J., Liu, W., Zhang, X.-P. S., & Mei, T. (2021) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas51556.2021.9401218 · Cited by 19
“Inspired by curriculum learning [1], we refine the backbone CNN in a from-easy-to-hard manner, by which we train the backbone in R rounds.”
Liu, N., Lu, T., Cai, Y., Wang, R., & Wang, S. (2021) · Science China Information Sciences · doi:10.1007/s11432-019-2648-7 · Cited by 12
“Curriculum learning [4], which starts learning small domains with easier aspects of tasks and then gradually increases the difficulty level, is also applied to learn relatively easy manipulation skills [5].”
BackgroundFast-Learning Grasping and Pre-Grasping via Clutter Quantization and Q-map Masking
Ren, D., Ren, X., Wang, X., Digumarti, S. T., & Shi, G. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636165 · Cited by 12
“First of all, inspired by the idea of curriculum learning [26] and domain randomization [27], we randomly”
BackgroundGradient-based Algorithms for Machine Teaching
Wang, P., Nagrecha, K., & Vasconcelos, N. M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00144 · Cited by 19
“Note that there are some related algorithm families to machine teaching, including active learning [37, 45], few-shot learning [42, 41], curriculum learning [4, 14] and knowledge distillation [17].”
BackgroundMethodIndoor Path Planning for an Unmanned Aerial Vehicle via Curriculum Learning
Park, J. M., Jang, S., & Shin, Y. (2021) · International Conference on Control, Automation and Systems (ICCAS) · doi:10.23919/iccas52745.2021.9649794 · Cited by 11
“This technique offers the advantages of generalization and a fast convergence speed [42].”
Improving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning
Zhang, X., Yu, B., Liu, T., Zhang, Z., Sheng, J., Xue, M., & Xu, H. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.839 · Cited by 18
“As studied in previous works (Bengio et al., 2009; Arpit et al., 2017), hard samples can not be learnt effectively at first, thus predictions of those mislabeled hard samples may not fluctuate and then they are mistakenly believed to be reliable.”
BackgroundBurnt Forest Estimation from Sentinel-2 Imagery of Australia using Unsupervised Deep Learning
Abid, N., Malik, M. I., Shahzad, M., Shafait, F., Ali, H., Ghaffar, M. M., Weis, C., Wehn, N., & Liwicki, M. (2021) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta52665.2021.9647174 · Cited by 13
A Self-paced Learning based Transfer Model for Hypergraph Matching
Zhu, H., Wang, X., Xu, G., & Deng, L. (2022) · Information Sciences · doi:10.1016/j.ins.2022.01.005 · Cited by 11
Engaging decolonization: lessons toward a theory of change for transforming practice
Wilson, C. E. (2022) · Teaching in Higher Education · doi:10.1080/13562517.2022.2053672 · Cited by 14
“This primarily includes content, planning, and documentation, but also the system through which decisions are made and concepts are explored (Bengio et al. 2009).”
BackgroundROI-Constrained Bidding via Curriculum-Guided Bayesian Reinforcement Learning
Wang, H., Du, C., Fang, P., Yuan, S., He, X., Wang, L., & Zheng, B. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539211 · Cited by 24
“Curriculum Learning has been explored on supervised learning [4], and is extended to RL in [21, 38] to deal with sparse reward.”
BackgroundESTIMATING TIME-VARYING REPRODUCTION NUMBER BY DEEP LEARNING TECHNIQUES
Song, P., & Xiao, Y. (2022) · Journal of Applied Analysis & Computation · doi:10.11948/20220136 · Cited by 13
Liew, A., Lee, C. C., Subramaniam, V., Lan, B. L., & Tan, M. (2022) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.28456 · Cited by 12
“32,33 With CAVEAT, the aim was to create small shifts in domain so that the network can adapt to the target domain.”
BackgroundKNNGAN: an oversampling technique for textual imbalanced datasets
Madani, M., Motameni, H., & Mohamadi, H. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-022-04851-3 · Cited by 12
“To adjust these hyperparameters in the present paper, the Random Search [85] was used, which is one of the widely used hyperparameter optimization techniques.”
MethodInteractive Robot Learning: An Overview
Chétouani, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-24349-3_9 · Cited by 21
“Organization of a training strategy is termed curriculum learning [8], which is a key element in human teaching.”
Kesgin, H. T., & Amasyalı, M. F. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3265331 · Cited by 18
“For example, object number [6] and shape variability [2] for image datasets, sentence length [7], and word rarity [8] for text datasets.”
BackgroundMethodInfluentialA multi-agent decision approach for optimal energy allocation in microgrid system
Huang, M., Lin, X., Feng, Z., Wu, D., & Shi, Z. (2023) · Electric Power Systems Research · doi:10.1016/j.epsr.2023.109399 · Cited by 11
On The Design and Training Strategies for Rnn-Based Online Neural Speech Separation Systems
Li, K., & Luo, Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096724 · Cited by 13
Artificial Intelligence and Machine Learning in Drug Discovery and Development
Johari, R., Gupta, A., Sharma, A., Garg, S., Nagarajan, K., & Bhatt, P. (2023) · International Conference on System Modeling & Advancement in Research Trends (SMART) · doi:10.1109/smart59791.2023.10428489 · Cited by 20
Supervised Learning with Minimal Effort
Ni, E. A., & Ling, C. X. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13672-6_45 · Cited by 11
Textual Similarity with a Bag-of-Embedded-Words Model
Clinchant, S., & Perronnin, F. (2013) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/2499178.2499180 · Cited by 12
Harnessing noisy Web images for deep representation
Vo, P. D., Ginsca, A., Borgne, H. L., & Popescu, A. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.01.009 · Cited by 22
“The approach of Chen and Gupta (2015) is inspired by curriculum learning ( Bengio et al., 2009a ) applied to webly su- Please cite this article as: P.D. Vo et al., Harnessing noisy Web images fo (2017), http://dx.doi.org/10.1016/j.cviu.2017.01.009 ervised learning of CNN.”
MethodInfluentialSelf-paced Convolutional Neural Network for Computer Aided Detection in Medical Imaging Analysis
Li, X., Zhong, A., Lin, M., Guo, N., Sun, M., Sitek, A., Ye, J., Thrall, J. H., & Li, Q. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67389-9_25 · Cited by 14
“As the name implies, our spCNN framework is motivated by self-paced learning [6, 7, 8] and curriculum learning [9] methods, which adaptively choose part of labeled instances for the training.”
MethodВячеславович, Е. М., & Tiumentsev, Y. V. (2018) · Procedia Computer Science · doi:10.1016/j.procs.2018.01.022 · Cited by 11
MSnet: Multi-Head Self-Attention Network for Distantly Supervised Relation Extraction
Sun, T., Zhang, C., Ji, Y., & Hu, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2913316 · Cited by 12
“[18] which utilizes curriculum learning to model the noise in the training data, we become interested in curriculum learning [19] and selfpaced learning [20], [21].”
BackgroundMethodCount, Crop and Recognise: Fine-Grained Recognition in the Wild
Bain, M., Nagrani, A., Schofield, D. P., & Zisserman, A. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00032 · Cited by 18
“By tackling an easier task first, our model is using a form of curriculum learning [3].”
MethodIncremental Training of a Recurrent Neural Network Exploiting a Multi-scale Dynamic Memory
Carta, A., Sperduti, A., & Bacciu, D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67658-2_39 · Cited by 11
“itdoesnotrequiretheexpensivepretrainingofthe unrolledmodel.Furthermore,theincrementalapproachislesscomputationally expensiveforlongsequences. Theincrementaltrainingcanbeseenasaformofcurriculumtraining[2] sincethemodelistrainedongraduallymoredifficultproblemsbyincreasing thecomplexityofthenetwork.Themaindifferenceisthattr…”
A New Three-stage Curriculum Learning Approach for Deep Network Based Liver Tumor Segmentation
Li, H., Liu, X., Boumaraf, S., Liu, W., Gong, X., & Ma, X. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206789 · Cited by 14
Self-paced learning enhanced neural matrix factorization for noise-aware recommendation
Liu, Z., Feng, X., Wang, Y., & Zuo, W. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106660 · Cited by 14
Controllable and Progressive Image Extrapolation
Li, Y., Jiang, L., & Yang, M. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00219 · Cited by 20
“The line of research work generally regards finding an optimal order of executing some known tasks [3].”
BackgroundDo We Really Need Gold Samples for Sample Weighting under Label Noise?
Ghosh, A., & Lan, A. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00397 · Cited by 15
“A similar concept is introduced in curriculum learning where model learns from easy samples first, and then adapts to harder examples [6, 29, 11, 36].”
BackgroundDuminy, N., Nguyen, S. M., Zhu, J., Duhaut, D., & Kerdreux, J. (2021) · Applied Sciences · doi:10.3390/app11030975 · Cited by 19
“This approach has been coined ’curriculum learning’ in [4].”
Ma, C., Li, D., & Jia, X. (2020) · Cited by 11
Unsupervised Medical Image Alignment With Curriculum Learning
Burduja, M., & Ionescu, R. T. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506067 · Cited by 21
Öztürk, A., Gunel, M. B., Dagdanov, R., Vural, M. E., Yurdakul, F., Dal, M., & Üre, N. K. (2021) · IEEE Intelligent Vehicles Symposium Workshops (IV Workshops) · doi:10.1109/ivworkshops54471.2021.9669203 · Cited by 15
“One of the earlier successful examples of using this methodology in machine learning is the Projection Problem study by Elman [5] and observation of the effects of curriculum learning on training time and effectiveness by Bengio [6].”
MethodWhispered and Lombard Neural Speech Synthesis
Hu, Q., Bleisch, T., Petkov, P. N., Raitio, T., Marchi, E., & Lakshminarasimhan, V. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383454 · Cited by 19
“Similarly, in low-resource environments, the pre-training/finetuning model [25, 26] is utilized to adapt to each style.”
BackgroundKnowledge Transfer with Weighted Adversarial Network for Cold-Start Store Site Recommendation
Liu, Y., Guo, B., Zhang, D., Zeghlache, D., Chen, J., Hu, K., Zhang, S., Zhou, D., & Yu, Z. (2021) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3442203 · Cited by 17
“Inspired by curriculum learning [1], which organizes examples by a better arrangement to promote stronger optimization, we aim to select high-quality source data and transfer them to reduce negative transfer.”
MethodKuse, M., & Shen, S. (2021) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2021.103813 · Cited by 17
“[54], it was observed that for faster convergence, it is crucial to select triplets from the training dataset, that violate the triplet constraint, ie.”
BackgroundKim, H., & Lim, Y.-K. (2021) · Designing Interactive Systems Conference · doi:10.1145/3461778.3462141 · Cited by 17
“Bengio et al. [9] have been inspired by Skinner’s idea of shaping in exploring the idea of curriculum learning for ML algorithms.”
BackgroundCurriculum Learning-Based Artificial Neural Network Model for Solving Differential Equations
Sahoo, A. K., & Chakraverty, S. (2021) · Studies in computational intelligence · doi:10.1007/978-981-16-4713-0_6 · Cited by 12
An empirical study of cyclical learning rate on neural machine translation
Wang, W., Lee, C. M., Liu, J., Çolakoğlu, T., & Peng, W. (2022) · Natural Language Engineering · doi:10.1017/s135132492200002x · Cited by 14
Deep Curriculum Learning for PolSAR Image Classification
Mousavi, H., Imani, M., & Ghassemian, H. (2022) · Iranian Conference on Machine Vision and Image Processing · doi:10.1109/mvip53647.2022.9738781 · Cited by 13
“Curriculum Learning (CL) can be considered as an optimization technique, which can improve optimization performance by discovering better relative minima of a non-convex cost function [5], [7].”
BackgroundCLOSE: Curriculum Learning on the Sharing Extent Towards Better One-Shot NAS
Zhou, Z., Ning, X., Cai, Y., Han, J., Deng, Y., Dong, Y., Yang, H., & Wang, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20044-1_33 · Cited by 19
Emergent proximo-distal maturation through adaptive exploration
Stulp, F., & Oudeyer, P. (2012) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2012.6400586 · Cited by 10
“This includes mechanisms of intrinsically motivated active learning, which automatically select training examples or tasks of increasing complexity [16], [2], social learning where a teacher can scaffold a training program of increasing difficulty [11] and also curriculum learning [3], as well as maturation and body g…”
BackgroundEvaluation of Deep Learning on an Abstract Image Classification Dataset
Stabinger, S., & Rodríguez-Sánchez, A. J. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.325 · Cited by 12
“[1] under the name of curriculum learning.”
Representation Matters: Improving Perception and Exploration for Robotics
Wulfmeier, M., Byravan, A., Hertweck, T., Higgins, I., Gupta, A., Kulkarni, T. D., Reynolds, M., Teplyashin, D., Hafner, R., Lampe, T., et al. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560733 · Cited by 21
“Random sampling often provides a strong baseline [53] but can be improved via curriculum learning [50], [54]–[56] as well as iterative generation of tasks [57], [58].”
BackgroundExtensions and Limitations of the Neural GPUPreprint
Price, E., Zaremba, W., & Sutskever, I. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00736 · Cited by 21
“In particular, all the aforementioned models require a curriculum learning (Bengio et al., 2009; Zaremba & Sutskever, 2014) in order to successfully learn sophisticated functions, and the results in this paper are no different.”
BackgroundUnbounded cache model for online language modeling with open vocabularyPreprint
Grave, E., Cissé, M., & Joulin, A. (2017) · Neural Information Processing Systems · doi:10.48550/arxiv.1711.02604 · Cited by 102
“Notable variants of transfer learning are multitask learning [10], domain adaptation [6], and curriculum learning [8].”
BackgroundMultiple Instance Curriculum Learning for Weakly Supervised Object DetectionPreprint
Li, S., Zhu, X., Huang, Q., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.09191 · Cited by 11
“[3], indicating that learning from easy to hard examples can be beneficial.”
BackgroundMethodModeling Coherence for Discourse Neural Machine TranslationPreprint
Xiong, H., He, Z., Wu, H. R., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.05683 · Cited by 11
“cattered, ensuring the effectiveness of our other systems since they are all trained like the first-pass system. In addition, this training strategy can be viewed as well designed curriculum learning (Bengio et al. 2009) strategy, which has been proved effective for NMT task (Kocmi and Bojar 2017). Compared to the first…”
MethodImproving MAE against CCE under Label Noise.Preprint
Wang, X., Kodirov, E., Yang, H., & Robertson, N. M. (2019) · arXiv (Cornell University) · Cited by 16
Curiosity-Driven Multi-Criteria Hindsight Experience ReplayPreprint
Lanier, J. B., McAleer, S., & Baldi, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.03710 · Cited by 17
“Previous applications of curriculum learning (Bengio et al., 2009) to goal based environments include training on a variety of tolerances for considering goals achieved (Fournier et al., 2018), masking certain goal dimensions to allow all such values on an axis to be sufficient for success (Eppe et…”
BackgroundUnderwater object detection using Invert Multi-Class Adaboost with deep learningPreprint
Chen, L., Liu, Z., Tong, L., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.11552 · Cited by 11
“For Faster RCNN, we use three backbone networks including VGG16, ResNet50 [34] and ResNet101 [34].”
BackgroundMethodInfluential
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.