Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 1,801–1,900 of 5,240
Improving Medical Multi-modal Contrastive Learning with Expert Annotations
Kumar, Y., & Marttinen, P. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72661-3_27 · Cited by 43
Time and frequency synergy for source-free time-series domain adaptations
Furqon, M. T., Pratama, M., Shiddiqi, A. M., Liu, L., Habibullah, H., & Doğançay, K. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121734 · Cited by 16
Bouamrane, A., Derdour, M., Bennour, A., et al. (2024) · Diagnostics · doi:10.3390/diagnostics15010001 · Cited by 13
Ramezankhani, M., Deodhar, A., Parekh, R. Y., & Birru, D. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109886 · Cited by 26
“To mitigate this issue, one effective approach involves utilizing curriculum learning strategies (Krishnapriyan et al., 2021; Bengio et al., 2009).”
Comparison of Leading AI Models an Analytical Study of ChatGPT Google Bard and Microsoft Bing
Adomako, P., Khan, T. A., Ali, R. H., & Koutaly, R. (2025) · ADCAIJ ADVANCES IN DISTRIBUTED COMPUTING AND ARTIFICIAL INTELLIGENCE JOURNAL · doi:10.14201/adcaij.31857 · Cited by 13
Machine Learning for Holistic Evaluation of Scientific Essays
Hughes, S., Hastings, P., Britt, M. A., Wallace, P. S., & Blaum, D. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19773-9_17 · Cited by 13
“Curriculum learning is the technique of training machine learning models on gradually harder training examples, and lowers the generalization error in some problems [1].”
MethodSynchronization for multi-perspective videos in the wild
Liang, J., Huang, P.-Y., Chen, J., et al. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952425 · Cited by 13
Source-Aware Context Network for Single-Channel Multi-Speaker Speech Separation
Li, Z.-X., Song, Y., Dai, L.-R., et al. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461578 · Cited by 13
“2, a strategy was employed following curriculum learning [21]: The training starts from small S in Eqn.”
MethodCurriculum Design for Machine Learners in Sequential Decision Tasks
Peng, B., MacGlashan, J., Loftin, R., Littman, M. L., Roberts, D. L., & Taylor, M. E. (2018) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2018.2829980 · Cited by 14
“[6], curricula were used for non-convex optimization problems in machine learning.”
Pareto Self-Paced Learning Based on Differential Evolution
Li, H., Gong, M., Wang, C., et al. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2935762 · Cited by 13
“To solve this problem, Bengio et al. [1] proposed a learning paradigm called curriculum learning (CL) to simulate the learning rules of humans/animals in which they learn the easy concepts at first and then increasingly learn more complex knowledge.”
BackgroundAlternative Loss Functions in AlphaZero-like Self-play
Wang, H., Emmerich, M., Preuß, M., & Plaat, A. (2019) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci44817.2019.9002814 · Cited by 11
“[2] turned to self-play to generate training data instead of training from human data (AlphaGo Zero), which not only saves a lot of work of collecting and labeling data from human experts, but also shifts the constraining factor for learning from available data to computing power, and achieves a form of efficient curr…”
BackgroundWeakly supervised easy-to-hard learning for object detection in image sequences
Yu, H., Guo, D., Yan, Z., Fu, L., Simmons, J. P., Przybyla, C., & Wang, S. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.02.075 · Cited by 14
“This easy-to-hard learning process is similar as Curriculum Learning [12] and Baby-Steps Curriculum Learning [13] .”
BackgroundDifferentiable Adaptive Computation Time for Visual Reasoning
Eyzaguirre, C., & Soto, Á. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01283 · Cited by 22
“Estimators such as the average complexities of questions can be useful for curriculum learning [3], providing a novel way of separating data that does not require supervision.”
BackgroundRelabel the Noise: Joint Extraction of Entities and Relations via Cooperative Multiagents
Chen, D., Li, Y., Lei, K., & Shen, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.527 · Cited by 14
“In this study, we extend the curriculum learning strategy (Bengio et al., 2009) to our cooperative multia-gents.”
MethodAverage Top-k Aggregate Loss for Supervised Learning
Lyu, S., Fan, Y., Ying, Y., & Hu, B.-G. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3005393 · Cited by 13
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering
Xiong, W., Wang, H., & Wang, W. Y. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.244 · Cited by 20
“Note that our training algorithm shares spirits with Curriculum Learning (Bengio et al., 2009) and SelfPaced Learning (Kumar et al.”
Reducing BERT Computation by Padding Removal and Curriculum Learning
Zhang, W., Wei, W., Wang, W., Jin, L., & Cao, Z. (2021) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass51385.2021.00025 · Cited by 22
“To address this issue, this work explores CL [3], [4] to train the model with the many buckets.”
MethodCho, Y., Park, B., Lee, S. M., Lee, K. H., Seo, J. B., & Kim, N. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104750 · Cited by 13
DRQN-based 3D Obstacle Avoidance with a Limited Field of View
Chen, Y., Chen, G., Pan, L., Ma, J., Zhang, Y., Zhang, Y., & Ji, J. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635949 · Cited by 14
“Curriculum learning [23] has recently been shown as an efficient strategy that can help find better local minima and accelerate the training process.”
BackgroundAutoFac: The Perpetual Robot Machine
Nitschke, G., & Howard, D. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3104789 · Cited by 16
Density-Based Dynamic Curriculum Learning for Intent Detection
Gong, Y., Liu, C., Yuan, J., et al. (2021) · Proceedings of the ... ACM International Conference on Information & Knowledge Management · doi:10.1145/3459637.3482082 · Cited by 13
An instance-oriented performance measure for classification
Yu, S., Li, X., Feng, Y., Zhang, X., & Chen, S. (2021) · Information Sciences · doi:10.1016/j.ins.2021.08.094 · Cited by 13
A Deep Learning Approach to Capture the Essence of Candida albicans Morphologies
Bettauer, V., Costa, A. C. B. P., Omran, R. P., Massahi, S., Kirbizakis, E., Simpson, S., Dumeaux, V., Law, C. O., Whiteway, M. S., & Hallett, M. T. (2022) · Microbiology Spectrum · doi:10.1128/spectrum.01472-22 · Cited by 15
PourNet: Robust Robotic Pouring Through Curriculum and Curiosity-based Reinforcement Learning
Babaians, E., Sharma, T., Karimi, M., Sharifzadeh, S., & Steinbach, E. G. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981195 · Cited by 21
“Curriculum Learning [20] is inspired from human behavior of breaking down a complex difficult task into smaller structured tasks with increasing difficulty.”
BackgroundKINet: Incorporating Relevant Facts Into Knowledge-Grounded Dialog Generation
Bai, J., Yang, Z., Yang, J., Guo, H., & Li, Z. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3240654 · Cited by 18
“The curriculum knowledge sampler is proposed to sample relevant knowledge pieces over the approximate knowledge distribution in a curriculum learning manner [17], [18].”
BackgroundHow Many Annotations Do We Need for Generalizing New-Coming Shadow Images?
Wu, W., Yang, W., Ma, W., & Chen, X.-D. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3263903 · Cited by 14
“To this end, we successively introduce two solutions to improve the quality of pseudo masks for them: (1) Exploiting the whole unlabeled data from easy to hard, which is also known as curriculum learning [27]; (2) further providing weak labels for these unseen images.”
BackgroundPeano: learning formal mathematical reasoning
Poesia, G., & Goodman, N. D. (2023) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2022.0044 · Cited by 35
“This schema matches a well-known definition of a training curriculum [3]: a function that assigns weights to training examples at each iteration in such a way that, at the end of training, sampling from the reweighted distribution is equivalent to sampling from the original target distribution, but examples might be g…”
BackgroundWhich Samples Should Be Learned First: Easy or Hard?
Zhou, X., & Wu, O. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3284430 · Cited by 22
“CL [25], [29] is motivated by human learning that easy samples should be learned first.”
BackgroundStable skill improvement of quadruped robot based on privileged information and curriculum guidance
Jiang, H. X., Chen, T., Cao, J., Bi, J. Q., Lu, G., Zhang, G., Rong, X., & Li, Y. (2023) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2023.104550 · Cited by 17
Tag-based video retrieval by embedding semantic content in a continuous word space
Agharwal, A., Kovvuri, R., Nevatia, R., & Snoek, C. G. M. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477706 · Cited by 15
“Authors in [8] propose a re-ranking scheme for multi-modal data inspired from curriculum learning methods proposed in [1, 12].”
RNN Based Uyghur Text Line Recognition and Its Training Strategy
Li, P., Zhu, J., Peng, L., & Guo, Y. (2016) · IAPR International Workshop on Document Analysis Systems (DAS) · doi:10.1109/das.2016.20 · Cited by 13
“developed curriculum learning concept in [10], which is to start training a simple subset and building more and more sophisticated representations of structure.”
BackgroundModeling sensory-motor decisions in natural behavior
Zhang, R., Zhang, S., Tong, M. H., Cui, Y., Rothkopf, C. A., Ballard, D. H., & Hayhoe, M. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006518 · Cited by 15
Accelerating the Deep Reinforcement Learning with Neural Network Compression
Zhang, H., He, Z., & Li, J. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852451 · Cited by 20
“In Mix & Match, they used curriculum learning [18] on agent instead of environment.”
MethodDeep Convolutional Neural Network Using Triplet Loss to Distinguish the Identical Twins
Ahmad, B., Usama, M., Lu, J., Xiao, W., Wan, J., & Yang, J. (2019) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshps45667.2019.9024704 · Cited by 16
“The most significant task is to select those triplets, which able to achieve good effectiveness and motivated by curriculum learning [12].”
BackgroundKim, T., & Lee, J.-H. (2020) · IEEE Access · doi:10.1109/access.2020.3015245 · Cited by 15
“However, most previous work manually determines the curriculum without considering the maturity of policy learning [49]–[52].”
Multi-Robot Cooperative Target Encirclement through Learning Distributed Transferable Policy
Zhang, T., Liu, Z., Wu, S., Pu, Z., & Yi, J. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207248 · Cited by 15
“Formalizing such training strategies calls curriculum learning [21].”
BackgroundCap2Seg: Inferring Semantic and Spatial Context from Captions for Zero-Shot Image Segmentation
Tian, G., Wang, S., Feng, J., Zhou, L., & Mu, Y. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413990 · Cited by 16
Simultaneous Planning for Item Picking and Placing by Deep Reinforcement Learning
Tanaka, T., Kaneko, T., Sekine, M., Tangkaratt, V., & Sugiyama, M. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340929 · Cited by 14
Reinforcement Learning based Negotiation-aware Motion Planning of Autonomous Vehicles
Wang, Z., Zhuang, Y., Gu, Q., Chen, D., Zhang, H., & Liu, W. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635935 · Cited by 14
“As for the various of interactive situations, we leveraged curriculum learning to handle it [29].”
BackgroundGranskog, J., Schnabel, T. N., Rousselle, F., & Novák, J. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459848 · Cited by 21
Implementation of Decentralized Reinforcement Learning-Based Multi-Quadrotor Flocking
Abichandani, P., Speck, C., Bucci, D. J., Mcintyre, W., & Lobo, D. (2021) · IEEE Access · doi:10.1109/access.2021.3115711 · Cited by 13
“In [75], the authors showed that starting the training procedure with easier examples of a learning task followed by a gradual increase in the difficulty improved the speed of convergence and the generalizability of the results.”
BackgroundTravelNet: Self-supervised Physically Plausible Hand Motion Learning from Monocular Color Images
Zhao, Z., Zhao, X., & Wang, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01146 · Cited by 20
“To add jitter to the motion, a series of jitter masks Mi(λ, τ) ∈ R21×N are created in a curriculum learning scheme [6] and element-wise product is conducted on ΘN :”
MethodOvercoming Challenges of Applying Reinforcement Learning for Intelligent Vehicle Control
Pina, R., Tibebu, H., Hook, J., Silva, D., & KONDOZ, A. M. (2021) · Sensors · doi:10.3390/s21237829 · Cited by 13
“Reference [36] defines CL(CL) as a sequence of learning distributions Q(λ), 0 ≤ λ ≤ 1, where: ∀e > 0 : λ < λ + e→ H(Qλ) < H(Qλ+e),”
BackgroundMethodInfluentialDeep Learning Logo Detection with Data Expansion by Synthesising ContextPreprint
Su, H., Zhu, X., & Gong, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.09322 · Cited by 13
“Curriculum Learning (CL), designed for reducing the difficulties in learning tasks by easy-to-hard staged learning [4].”
BackgroundMethodLearning knowledge to support domain-independent narrative intelligence
Li, B. (2015) · SMARTech Repository (Georgia Institute of Technology) · Cited by 15
Capturing Long-range Contextual Dependencies with Memory-enhanced Conditional Random FieldsPreprint
Liu, F., Baldwin, T., & Cohn, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.03637 · Cited by 15
“mand ←− W m ∈R190×190, and b ∈R190. Dropout is applied to all GRU recurrent units on the input and output connections with a keep rate of 0.7. Lastly, we also explore the idea of curriculum learning (Bengio et al., 2009), by fixing the CRF transition matrix A = 0 for the first e = 20 epochs, after whichwetrain theparame…”
BackgroundMethodInfluentialDenoising Neural Machine Translation Training with Trusted Data and Online Data SelectionPreprint
Wang, W., Watanabe, T., Hughes, M., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00068 · Cited by 13
“es not seem to improve. Previous work on domain data selection has shown that the order in which data are scheduled matters a lot for NMT training, a research that is relevant to curriculum learning (Bengio et al., 2009) in machine learning literature. (van der Wees et al., 2017) show the effectiveness of a nice “grad…”
BackgroundInfluentialSkynet: A Top Deep RL Agent in the Inaugural Pommerman Team CompetitionPreprint
Gao, C., Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.01360 · Cited by 20
“Curriculum Learning Training is conducted by letting two identical neural net players compete against a set of curriculum [1] opponents: (i) Static opponent teams, where opponents do not move or place bombs.”
BackgroundTeaching Deep Convolutional Neural Networks to Play GoPreprint
Clark, C., & Storkey, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.3409 · Cited by 41
“Curriculum learning (Bengio et al., 2009) and integration with reinforcement learning techniques might provide avenues for improvement.”
BackgroundSelf-Paced Contextual Reinforcement LearningPreprint
Klink, P., Abdulsamad, H., Belousov, B., & Peters, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.02826 · Cited by 22
“Indeed, advances in gradual and sequential learning, exemplified by shaping [14] and curriculum learning [15], which are loosely based on the insights from behavioral psychology [16] and continuation methods in optimization [17], demonstrate significant improvements in the speed of learning and…”
BackgroundCurriculum Learning in Deep Neural Networks for Financial ForecastingPreprint
Koenecke, A., & Gajewar, A. (n.d.) · RePEc: Research Papers in Economics · Cited by 13
Navigating the landscape of multiplayer games
Omidshafiei, S., Tuyls, K., Czarnecki, W. M., Santos, F. C., Rowland, M., Connor, J. T., Hennes, D., Müller, P. F., Pérolat, J., Vylder, B. D., et al. (2020) · RePEc: Research Papers in Economics · Cited by 12
Sampling methods for efficient training of graph convolutional networks: A surveyPreprint
Liu, X., Yan, M., Deng, L., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.05872 · Cited by 13
“The authors also leverage some tricks, such as negative sampling [63], to optimize the PinSage-based training process and further design a curriculum training approach [64] for faster convergence.”
MHER: Model-based Hindsight Experience ReplayPreprint
Yang, R., Fang, M., Han, L., Du, Y., Luo, F., & Li, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.00306 · Cited by 35
“Previous works have proposed many solutions such as reward shaping [22], curriculum learning [3], exploration [23, 34], and hindsight relabeling [16, 1].”
BackgroundPhysFormer: Facial Video-based Physiological Measurement with Temporal Difference TransformerPreprint
Yu, Z., Shen, Y., Shi, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12082 · Cited by 13
“Curriculum learning [2], as a major machine learning regime with philosophy of easy-to-hard curriculum, is utilized to train PhysFormer.”
MethodRobotics: Science and Systems XVII
(2021) · doi:10.15607/rss.2021.xvii · Cited by 13
Vlachas, P. R., & Koumoutsakos, P. (2024) · Physica D Nonlinear Phenomena · doi:10.1016/j.physd.2024.134371 · Cited by 21
Social robot navigation: a review and benchmarking of learning-based methods
Alyassi, R., Cadena, C., Riener, R., & GRANADOS, D. F. P. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1658643 · Cited by 25
“…rate or adaptively based on agent performance, and may involve modifying reward functions or start/goal distributions (Riedmiller et al., 2018; Florensa et al., 2018), optimization strategies (Matiisen et al., 2019), Curriculum MDPs (Narvekar et al., 2017), or human feedback (Bengio et al., 2009).”
Self-Training With Double Selectors for Low-Resource Named Entity Recognition
Fu, Y., Lin, N., Yu, X., & Jiang, S. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3250828 · Cited by 11
“Multiple methods have been proposed to select easy samples [22].”
MethodCGF: A Category Guidance Based PM$_{2.5}$ Sequence Forecasting Training Framework
Yu, H., Hu, J., Zhou, X., Guo, C., Yang, B., & Li, Q. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3253703 · Cited by 13
“Then, self-paced learning (SPL) [21] is proposed to select easy samples automatically.”
BackgroundMethodInfluentialSelf-Supervised Training of Speaker Encoder With Multi-Modal Diverse Positive Pairs
Tao, R., Lee, K. A., Das, R. K., Hautamäki, V., & Li, H. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3268568 · Cited by 18
“Progressive clustering is motivated by the curriculum learning principle, an efficient strategy in machine learning by increasing the difficulty of training gradually [42], [53].”
EDITS: An Easy-to-difficult Training Strategy for Cloud Failure Prediction
Lin, Q., Li, T., Zhao, P., Liu, Y., Ma, M., Zheng, L., Chintalapati, M., Liu, B., Wang, P. P., Zhang, H., et al. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543873.3584630 · Cited by 12
“As illustrated in curriculum learning [3], human and animals learn much better when the examples are not randomly presented, but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones.”
BackgroundLearning scalable and efficient communication policies for multi-robot collision avoidance
Serra-Gómez, Á., Zhu, H., Brito, B., Böhmer, W., & Alonso–Mora, J. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10127-3 · Cited by 18
“Hence, we employ curriculum learning (Bengio et al., 2009), training the agents first in a simple scenario, where communication is generally not needed, and subsequently introducing more difficult and complex scenarios where the agents must learn when and with whom to communicate.”
MethodAn Energy Matching Vessel Segmentation Framework in 3-D Medical Images
Liu, P., Huang, G. M., Jing, J., Bian, S., Cheng, L., Lu, X., Rao, C., Liu, Y., Hua, Y., Wang, Y., et al. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3339204 · Cited by 12
“To tackle this problem, Bengio et al. proposed in [20] the C urriculum L earning (CL) strategy, which is a training procedure that trains on simpler data subsets first, and then gradually increase the level of data difficulty.”
BackgroundSource Separation of Piano Concertos Using Musically Motivated Augmentation Techniques
Özer, Y., & Müller, M. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3356980 · Cited by 15
“…difficulty level gradually increases in the subsequent stages due to the rise in overlapping harmonics and onsets, this approach can be thought of as curriculum learning [77], which exploits, particularly in the first three stages, previously learned concepts to ease the learning of new abstractions.”
BackgroundDD-Net: Dual Decoder Network With Curriculum Learning for Full Waveform Inversion
Zhang, X.-Y., Min, F., Pan, S., Xu, Q., Min, X.-Y., Song, G., & Wang, K. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3358492 · Cited by 13
“Regarding the training process control, we introduce curriculum learning [27] to enhance network stability.”
BackgroundMethodA Time-Consistency Curriculum for Learning From Instance-Dependent Noisy Labels
Wu, S., Zhou, T., Du, Y., Yu, J., Han, B., & Liu, T. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3360623 · Cited by 15
“Curriculum Learning: Curriculum learning was first proposed by [22], which describes a learning paradigm in which a model is learned by gradually introducing samples of increasing hardness to training.”
BackgroundWu, Z., & Li, X. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3365784 · Cited by 12
“Besides the intuitive insight of learning from confident reconstructions, the mechanism behind our re-weighting method may have connections with curriculum learning [57] and active learning [58], which can be further explored in future works.”
BackgroundChen, K., Zhuang, D., Li, M., & Chang, J. M. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3396125 · Cited by 12
“from easy to difficult and gradually presents more difficult tasks to the NMT model during training (Bengio et al., 2009; Moore and Lewis, 2010; Wang et al., 2020; Zhan et al., 2021b).”
BackgroundMethodInfluentialMachine-learning strategies for the accurate and efficient analysis of x-ray spectroscopy
Penfold, T. J., Watson, L., Middleton, C., David, T. W., Verma, S., Pope, T. J., Kaczmarek, J., & Rankine, C. D. (2024) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ad5074 · Cited by 33
“In contrast, figure 17 (black) shows the effect of using curriculum learning (CL) [215, 216] to train the models.”
BackgroundMitigating Aberration-Induced Noise: A Deep Learning-Based Aberration-to- Aberration Approach
Sharifzadeh, M., Goudarzi, S., Tang, A., et al. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3422027 · Cited by 12
“Our interpretation suggests that the proposed loss function guides the optimizer towards a correct solution by initially utilizing simpler data, before gradually incorporating more complex, fluctuating RF data to take full advantage of the richer information, like curriculum learning [57].”
Mobile Robot Collision Avoidance Based on Deep Reinforcement Learning With Motion Constraints
Tao, Y., Li, M., Cao, X., & Lu, P. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3444854 · Cited by 16
“…training examples in a meaningful and progressively complex order, enhances learning efficiency and accelerates the training process by adjusting the difficulty and diversity of environments, outperforming other methods like Transfer of Learning, fine-tuning, and end-to-end training [25], [26], [27].”
Multimodal Emotion Recognition Calibration in Conversations
Tu, G., Xiong, F., Liang, B., Wang, H., Zeng, X., & Xu, R. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681515 · Cited by 36
“Our CMERC employs Curriculum Learning (CL) [1] to guide the model in progressively learning more uncertain samples that contradict the foundational principle in informatics, i.e., the removal of a modality or specific contextual cues enhances the model’s confidence, thus aiding in the model’s…”
BackgroundMethodDeep metric learning via subtype fuzzy clustering
Ren, C.-X., Li, J., Ge, P., et al. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.01.037 · Cited by 12
Improved reinforcement learning with curriculum
West, J., Maire, F., Browne, C., & Denman, S. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113515 · Cited by 13
“This approach resulted in an improved rate of learning [17] for a simple neural-network classifier.”
BackgroundCurriculum Learning in Deep Neural Networks for Financial Forecasting
Koenecke, A., & Gajewar, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-37720-5_2 · Cited by 12
“The relevant literature, including specifically the described Baby Steps algorithm [2,13], has been applied to LSTMs for parsing a Wall Street Journal corpus [13], n-gram language modeling [7], and for performing digit sums using LSTMs [4].”
MethodA Performance-Based Start State Curriculum Framework for Reinforcement Learning
Wöhlke, J., Schmitt, F., & van Hoof, H. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/llzi8531 · Cited by 23
“Curriculum learning has been shown to accelerate learning progress in supervised learning [6, 11, 17] as well as in reinforcement learning settings [8, 9, 19, 30].”
BackgroundDifferentiable Meta-Learning of Bandit Policies.
Boutilier, C. E., Hsu, C., Kveton, B., Mladenov, M., Szepesvári, C., & Zaheer, M. (2020) · neural information processing systems · Cited by 31
Son, J., Kim, J., Kong, S. T., & Jung, K.-H. (2021) · Applied Sciences · doi:10.3390/app11020591 · Cited by 12
Jafarpour, B., Sepehr, D., & Pogrebnyakov, N. (2021) · doi:10.18653/v1/2021.internlp-1.6 · Cited by 12
“CL borrows this idea and engages the human experts to design a metric that is used to sort the annotated training data from “easy” to “hard” to be presented to the model during training (Bengio et al. 2009).”
BackgroundGeneralization in Text-based Games via Hierarchical Reinforcement Learning
Xu, Y., Fang, M., Chen, L., Du, Y., & Zhang, C. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.116 · Cited by 22
“The scheduled task sampling is inspired by the curriculum learning (Bengio et al., 2009), where we schedule the tasks based on their difficulties.”
BackgroundSelf-Paced Curriculum Learning for Visual Question Answering on Remote Sensing Data
Yuan, Z., Mou, L., & Zhu, X. X. (2021) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss47720.2021.9553624 · Cited by 12
“Thus, incorporating prior knowledge is necessary to deal with this situation [10].”
BackgroundZhu, L., Wen, X., Lingqiang, M., Ma, J., & Wang, D. (2021) · Optik · doi:10.1016/j.ijleo.2021.168194 · Cited by 12
Domain adaptive twin support vector machine learning using privileged information
Li, Y., Sun, H., & Yan, W. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.10.069 · Cited by 12
Deep Learning for Object Detection in Materials-Science Images: A tutorial
Fu, L., Yu, H., Li, X., Przybyla, C., & Wang, S. (2021) · IEEE Signal Processing Magazine · doi:10.1109/msp.2021.3121558 · Cited by 13
“Similar to curriculum learning [28], this easy-to-hard learning method improves detection performance by iteratively including the 3D consistency prior in deep learning.”
MethodMiao, J., Yang, T., Fan, C. H., et al. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108279 · Cited by 12
Audio-Visual Gated-Sequenced Neural Networks for Affect Recognition
Aspandi, D., Sukno, F. M., Schuller, B. W., & Binefa, X. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3156026 · Cited by 15
“This is reached by employing LSTM combined with attention mechanisms [16] and training with Curriculum Learning [59], [62], [63].”
Generative Session-based Recommendation
Wang, Z., Ye, W., Chen, X., Zhang, W., Wang, Z., Zou, L., & Liu, W. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512095 · Cited by 18
“Another similar concept to SPL is curriculum learning (CL) [1], which also aims to train the model in an easy-to-hard manner.”
BackgroundConditional Generative Adversarial Networks for Domain Transfer: A Survey
Zhou, G., Fan, Y., Shi, J., Lu, Y., & Shen, J. (2022) · Applied Sciences · doi:10.3390/app12168350 · Cited by 16
Deep MAnTra: deep learning-based multi-animal tracking for Japanese macaques
Pineda, R. R., Kubo, T., Shimada, M., & Ikeda, K. (2022) · Artificial Life and Robotics · doi:10.1007/s10015-022-00837-9 · Cited by 15
“As in Bengio et al. [17, 18], we note faster convergence in succeeding stages in our 2-and 3-stage curriculum modules.”
BackgroundResultInfluentialDhakate, R., Brommer, C., Bohm, C., Gietler, H., Weiß, S., & Steinbrener, J. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981425 · Cited by 21
“For more complex manipulation tasks we plan to use curriculum learning [20], which has been shown to accelerate and improve the learning process.”
MethodVision UFormer: Long-range monocular absolute depth estimation
Polasek, T., Čadík, M., Keller, Y., & Beneš, B. (2022) · Computers & Graphics · doi:10.1016/j.cag.2023.02.003 · Cited by 12
Aerial Fluvial Image Dataset for Deep Semantic Segmentation Neural Networks and Its Benchmarks
Wang, Z., & Mahmoudian, N. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3275068 · Cited by 13
“Inspired by the concept of curriculum learning [87], where learning the broad concepts on less complex training sets first and then refining the concepts on more complex training sets gives distinct advantages over a network that is trained to grasp all scene complexities at once.”
BackgroundDiscrimLoss: A Universal Loss for Hard Samples and Incorrect Samples Discrimination
Wu, T., Ding, X., Zhang, H., Gao, J., Tang, M., Du, L., Qin, B., & Liu, T. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3290477 · Cited by 16
“To improve model performance and generalization, curriculum learning (CL) presents a new idea that easy samples should be learned before hard ones during training [12], [13].”
BackgroundRethinking Label Smoothing on Multi-Hop Question Answering
Yin, Z., Wang, Y., Hu, X., Wu, Y., Yan, H. R., Zhang, X., Cao, Z., Huang, X., & Qiu, X. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-6207-5_5 · Cited by 25
Stochastic co-teaching for training neural networks with unknown levels of label noise
de Vos, B. D., Jansen, G. E., & Išgum, I. (2023) · Scientific Reports · doi:10.1038/s41598-023-43864-7 · Cited by 19
FedCSS: Joint Client-and-Sample Selection for Hard Sample-Aware Noise-Robust Federated Learning
Li, A., Cao, Y., Guo, J., Peng, H., Guo, Q., & Yu, H. (2023) · Proceedings of the ACM on Management of Data · doi:10.1145/3617332 · Cited by 14
Multitask Kernel-based Learning with Logic Constraints
Diligenti, M., Gori, M., Maggini, M., et al. (2010) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-60750-606-5-433 · Cited by 12
“This also suggests the importance of devising appropriate teaching plans like the one exploited in curriculum learning [1].”
BackgroundResultContinuation Methods and Curriculum Learning for Learning to Rank
Ferro, N., Lucchese, C., Maistro, M., & Perego, R. (2018) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3269206.3269239 · Cited by 12
“In [2], Curriculum Learning is exploited to train a language model (not used for ranking) with a deep neural network.”
BackgroundInfluentialDeep Learning Methods for MRI Brain Tumor Segmentation: a comparative study
Brahim, I., Fourer, D., Vigneron, V., & Maaref, H. (2019) · International Conference on Image Processing Theory, Tools and Applications (IPTA) · doi:10.1109/ipta.2019.8936077 · Cited by 13
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.