Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 1,701–1,800 of 5,240
Choosing the Sample with Lowest Loss makes SGD RobustPreprint
Shah, V., Wu, X., & Sanghavi, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.03316 · Cited by 20
“Label noise in deep learning [2, 21, 4] describe different techniques to learn in presence of label noise and outliers.”
BackgroundEvaluating Logical Generalization in Graph Neural NetworksPreprint
Sinha, K., Sodhani, S., Pineau, J., & Hamilton, W. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.06560 · Cited by 33
“Thus, we perform an experiment following Curriculum Learning (Bengio et al., 2009) setup, where the order of the worlds being trained is determined by their relative difficulty (which is determined by the performance of models in supervised learning setup, Table 6, i.e., we order the worlds from…”
Differentiable Hierarchical Graph Grouping for Multi-Person Pose EstimationPreprint
Jin, S., Liu, W., Xie, E., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.11864 · Cited by 15
“The model learns to group from easy to hard, in a curriculum fashion [1].”
BackgroundGrounding Language to Entities and Dynamics for Generalization in Reinforcement Learning
Hanjie, A. W., Zhong, V., & Narasimhan, K. (2021) · arXiv · doi:10.48550/arxiv.2101.07393 · Cited by 15
“Thus, we introduce a three-stage curriculum to train our models (Bengio et al., 2009).”
MethodCarousel Memory: Rethinking the Design of Episodic Memory for Continual LearningPreprint
Lee, S., Weerakoon, M., Choi, J., Zhang, M., Wang, D., & Jeon, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07276 · Cited by 16
“This policy is motivated by curriculum learning (Bengio et al., 2009), which gradually focuses on training harder samples as time elapses.”
BackgroundCurriculum learning for language modeling
Campos, D. (2021) · arXiv · doi:10.48550/arxiv.2108.02170 · Cited by 15
“Experiments in modifying language modeling training data find a lower loss can be achieved by training on incrementally more difficult data [14].”
BackgroundEnd-to-End Autonomous Driving in CARLA: A Survey
Ozaibi, Y. A., Hina, M. D., & Ramdane-Chérif, A. (2024) · IEEE Access · doi:10.1109/access.2024.3473611 · Cited by 28
“Another possible solution to improve sample efficiency can be found in curriculum learning [179].”
BackgroundWang, Z., Zhan, W., Duan, H., & Huang, H. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110337 · Cited by 20
Wu, X., Tian, Y., Chen, Y., Ye, P., Cui, X., Jia, J., Li, S., Liu, J., & Niu, W. (2025) · Applied Sciences · doi:10.3390/app15169096 · Cited by 34
Baniya, S., & Maity, D. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11444-y · Cited by 26
Towards Self-organized Control: Using Neural Cellular Automata to Robustly Control a Cart-pole Agent
Variengien, A., Nichele, S., Glover, T. E., & Pontes-Filho, S. (2021) · doi:10.54854/imi2021.01 · Cited by 40
“This procedure is similar to what is used in curriculum learning [19] where an easy subset of the task is learned before tackling the whole problem.”
MethodSelf-Supervised Person Detection in 2D Range Data using a Calibrated Camera
Jia, D. P., Steinweg, M., Hermans, A., & Leibe, B. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561699 · Cited by 21
“Curriculum learning methods [38], [26] may help mitigating this problem, and thus relaxing the requirement of 0 0.”
MethodCatastrophic Forgetting Problem in Semi-Supervised Semantic Segmentation
Zhou, Y., Jiao, R., Wang, D., Mu, J., & Li, J. (2022) · IEEE Access · doi:10.1109/access.2022.3172664 · Cited by 14
RobustAnalog: Fast Variation-Aware Analog Circuit Design Via Multi-task RL
Shi, W., Wang, H., Gu, J., Liu, M., Pan, D. Z., Han, S., & Sun, N. (2022) · ACM/IEEE Workshop on Machine Learning for CAD (MLCAD) · doi:10.1145/3551901.3556487 · Cited by 52
“Optimized training task selections can significantly improve model performance [2].”
BackgroundA deep hashing method of likelihood function adaptive mapping
Su, H., Fang, J., Liu, W., Yu, S., & Yang, H. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07962-3 · Cited by 15
“Inspired by Curriculum learning [20] and multi-simi-larity loss approach that weighs easy and hard samples differently [21], this paper proposes a D eep A daptive-M apping-based H ashing (DAMH) method for likelihood function design 1 .”
MethodAutomatic Curriculum Learning for Large-Scale Cooperative Multiagent Systems
Zhang, T., Liu, Z., Pu, Z., & Yi, J. (2022) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2022.3209655 · Cited by 20
BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos
Sun, J. J., Karashchuk, P., Dravid, A. V., Ryou, S., Fereidooni, S., Tuthill, J. C., Katsaggelos, A. K., Brunton, B. W., Gkioxari, G., Kennedy, A., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00869 · Cited by 23
“Our model is trained using curriculum learning [1].”
MethodLeader learning loss function in neural network classification
Zhang, S., & Xie, L. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126735 · Cited by 15
Learning from Unique Perspectives: User-aware Saliency Modeling
Chen, S., Valliappan, N., Shen, S., Ye, X., Kohlhoff, K., & He, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00265 · Cited by 19
“Inspired by human learning that acquires knowledge through a set of carefully designed curricula [2], we propose to tackle the aforementioned issues with a progressive learning approach.”
BackgroundWu, J., Yang, H., Gan, T., Ding, N., Jiang, F., & Nie, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01513 · Cited by 24
“Towards this issue, FlexMatch [42] proposes a curriculum learning [4] strategy to learn adjustable class-specific threshold, which can well improve the results and efficiency.”
BackgroundPLACE Dropout: A Progressive Layer-wise and Channel-wise Dropout for Domain Generalization
Guo, J., Qi, L., Shi, Y., & Gao, Y. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3624015 · Cited by 24
“Such function is also utilized in curriculum learning and related progressive settings [2, 26].”
BackgroundHelvig, K., Trouvé-Peloux, P., Gavérina, L., Abeloos, B., & Roche, J.-M. (2023) · Quantitative InfraRed Thermography Journal · doi:10.1080/17686733.2023.2266176 · Cited by 24
“…the best of our knowledge, not in the context of FST. Curriculum learning, an approach to training neural networks that originates from behavioral psychology [17,18], consists in progressively introducing the machine learning system to more complex features through different training steps [19,20].”
Self-Paced Multi-Grained Cross-Modal Interaction Modeling for Referring Expression Comprehension
Miao, P., Su, W., Wang, G., Li, X., & Li, X. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3334099 · Cited by 22
“Different from curriculum learning [57], [58] that requires prior knowledge to determine the sample learning sequences, self-paced learning can dynamically realize sample learning during the training process.”
BackgroundMulti-Scenario and Multi-Task Aware Feature Interaction for Recommendation System
Song, D., Yang, E., Guo, G., Shen, L., Jiang, L., & Wang, X. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3651312 · Cited by 30
“Inspired by the ideas of curriculum learning [1] and progressively layered extraction [43], we design a middle layer to coarsely learn the differences in feature interaction importance between scenario–task pairs.”
MethodZhang, Y., Hu, H., Li, T., Chen, B., Tian, J., & Lei, S. (2024) · IEEE Transactions on Antennas and Propagation · doi:10.1109/tap.2024.3378385 · Cited by 14
“A self-paced learning (SPL) strategy is proposed to overcome this shortcoming [23].”
BackgroundQin, X., Li, X., Li, M., Zheng, H., & Xu, X. (2024) · The Visual Computer · doi:10.1007/s00371-024-03319-0 · Cited by 16
A Reinforcement Learning Approach to Military Simulations in Command: Modern Operations
Dimitriu, A., Michaletzky, T. V., Remeli, V., & Tihanyi, V. (2024) · IEEE Access · doi:10.1109/access.2024.3406148 · Cited by 18
Meta Reinforcement Learning of Locomotion Policy for Quadruped Robots With Motor Stuck
Chen, C., Li, C., Lu, H., Wang, Y., & Xiong, R. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2024.3424328 · Cited by 22
“Based on this range, we employ a curriculum training [32] method.”
A Comprehensive Study on Self-Learning Methods and Implications to Autonomous Driving
Xing, J., Wei, D., Zhou, S., Wang, T., Huang, Y., & Chen, H. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3440498 · Cited by 28
Steganalysis Network With Two-Branch Preprocessing for Spatial and JPEG Domains
He, J., Weng, S., Yu, L., & Chen, D. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3470809 · Cited by 19
“We adopt a curriculum learning [58] during training, namely 0.4 bpp → 0.3 bpp → 0.2 bpp → 0.1 bpp.”
InfluentialChallenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning
Fan, C., Liu, J., Hero, A. O., & Liu, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_16 · Cited by 70
Learning agility and adaptive legged locomotion via curricular hindsight reinforcement learning
Li, S., Wang, G., Pang, Y., Bai, P., Hu, S., Liu, Z., Wang, L., & Li, J. (2024) · Scientific Reports · doi:10.1038/s41598-024-79292-4 · Cited by 26
“A PD controller is used to calculate torque τ = Kp (ˆ q − q )+ Kd (ˆ˙ q − ˙ q ) , Kp and Kd are manually specified gains, which are set to 27.5 and 0.5 respectively.”
InfluentialSemantic video labeling by developmental visual agents
Gori, M., Lippi, M., Maggini, M., & Melacci, S. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.02.011 · Cited by 13
Academic Coupled Dictionary Learning for Sketch-based Image Retrieval
Xu, D., Alameda-Pineda, X., Song, J., et al. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2964329 · Cited by 14
“In parallel, curriculum learning (CL) [3] and self-paced learning (SPL) [18] strategies, have proven to be advantageous for various tasks in computer vision and multimedia, such as object tracking [32], event detection [16] or multimedia search [15].”
BackgroundODDS: Real-Time Object Detection Using Depth Sensors on Embedded GPUs
Mithun, N. C., Munir, S., Guo, K., & Shelton, C. (2018) · ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2018.00051 · Cited by 16
“Curriculum Learning: Curriculum learning is a training strategy [11, 41], where the complexity of training data is taken into account in training by distributing/dividing training samples based on complexity.”
Kamilati, N. (2018) · EDUKASI Jurnal Penelitian Pendidikan Agama dan Keagamaan · doi:10.32729/edukasi.v16i1.440 · Cited by 18
Cognitive Action Laws: The Case of Visual Features
Betti, A., Gori, M., & Melacci, S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2911174 · Cited by 15
“to a certain teaching plan (see [2], [3]).”
BackgroundThe teaching size: computable teachers and learners for universal languages
Telle, J. A., Hernández‐Orallo, J., & Ferri, C. (2019) · Machine Learning · doi:10.1007/s10994-019-05821-2 · Cited by 24
“…which is moving towards more general settings (Khan et al. 2011; Zhu 2013, 2015; Simard et al. 2017), and related areas such as curriculum learning (Bengio et al. 2009), understood as teaching with sequences of examples of increasing complexity or requiring from what has been learned before.”
BackgroundInteraction-Aware Multi-Agent Reinforcement Learning for Mobile Agents with Individual Goals
Mohseni-Kabir, A., Isele, D., & Fujimura, K. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793721 · Cited by 17
“As an alternative, we consider learning via a curriculum [9].”
MethodKeep off the Grass: Permissible Driving Routes from Radar with Weak Audio Supervision
Williams, D. S., Martini, D. D., Gadd, M., et al. (2020) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc45102.2020.9294415 · Cited by 14
“As described in [26], the idea of curriculum learning is that neural networks perform better when presented with the most understandable training examples first.”
BackgroundGenerating attentive goals for prioritized hindsight reinforcement learning
Liu, P., Bai, C., Zhao, Y., Bai, C., Zhao, W. Z., & Tang, X. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106140 · Cited by 14
Tracking-by-Trackers with a Distilled and Reinforced Model
Dunnhofer, M., Martinel, N., & Micheloni, C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69532-3_38 · Cited by 25
“s the policy loss and (7) is the value loss. These denitions follow the standard advantage actor-critic objective [48]. To further facilitate and improve the learning, a curriculum learning strategy [69] is built for each parallel student. During learning, the length of the interaction is increased as s performs bette…”
BackgroundInfluentialQayyum, A. B. A., Hassan, K., Anika, A., et al. (2020) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-020-00184-2 · Cited by 14
Learning to reweight examples in multi-label classification
Zhong, Y., Du, B., & Xu, C. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.03.022 · Cited by 14
Huang, T., Dilkina, B., & Koenig, S. (2021) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/tvdb7648 · Cited by 30
Liew, A., Lee, C. C., Lan, B. L., & Tan, M. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104690 · Cited by 15
“Curriculum learning advocates learning tasks with increasing difficulty to improve generalization of the network [40].”
BackgroundCompetence-based Curriculum Learning for Multilingual Machine Translation
Zhang, M., Meng, F., Tong, Y., & Zhou, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.212 · Cited by 24
“Curriculum learning was first proposed by Bengio et al. (2009) with the idea of learning samples from easy to hard to get a better optimized model.”
BackgroundDeep embeddings and logistic regression for rapid active learning in histopathological images
Jiao, Y., Yuan, J., Qiang, Y., & Fei, S. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106464 · Cited by 15
Zhang, J., Jia, X., & Hu, J. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3123268 · Cited by 17
Self-Supervised Curriculum Learning for Spelling Error Correction
Gan, Z., Xu, H., & Zan, H. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.281 · Cited by 14
“CL (Bengio et al., 2009) aims to facilitate the model training in an easy-to-hard order, which leads to improved model performance (Tsvetkov et al., 2016; Sachan and Xing, 2016; Amiri et al., 2017).”
BackgroundCurriculum learning for safe mapless navigation
Marzari, L., Corsi, D., Marchesini, E., & Farinelli, A. (2022) · ACM Symposium on Applied Computing · doi:10.1145/3477314.3507182 · Cited by 22
“For example, humans and animals are prone to learn better behaviors when meaningful experiences are not randomly encountered but are organized in a meaningful order which presents a new concept in increasing order of complexity [2].”
InfluentialCao, W., Li, S., & Zhong, J. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.07.005 · Cited by 18
Large-Scale ADMM-based Co-Design of Legged Robots
Bravo-Palacios, G., & Wensing, P. M. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981641 · Cited by 14
“The strategy follows curriculum-learning precepts [25].”
Wang, Q., Whitmarsh, S., Navarro, V., et al. (2022) · Proceedings of the VLDB Endowment · doi:10.14778/3570690.3570698 · Cited by 14
“The general idea of curriculum learning is to train models with increasing di � culty levels to bene � t convergence [7].”
BackgroundMethodOrdered racing protocols for automatically configuring algorithms for scaling performance
Styles, J., & Hoos, H. H. (2013) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2463372.2463438 · Cited by 14
Automated Curriculum Learning by Rewarding Temporally Rare Events
Justesen, N., & Risi, S. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490448 · Cited by 18
“Another approach is to gradually increase the difficulty of the environment to ease learning through curriculum learning [4, 46].”
BackgroundMethodKutsuzawa, K., Sakaino, S., & Tsuji, T. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2854958 · Cited by 16
“We herein applied curriculum learning [22].”
MethodSimultaneously Learning Vision and Feature-Based Control Policies for Real-World Ball-In-A-Cup
Schwab, D., Springenberg, J. T., Martins, M. F., Neunert, M., Lampe, T., Abdolmaleki, A., Hertweck, T., Hafner, R., Nori, F., & Riedmiller, M. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.027 · Cited by 27
“Just like SAC-X, our approach can also be related to curriculum learning approaches (see e.g. Bengio et al. [5] for a description and Heess et al. [12] for an RL application).”
A Neural State Pushdown Automata
Mali, A., Ororbia, A. G., & Giles, C. L. (2020) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3055167 · Cited by 17
“We also changed the number of hidden units for the baseline models but kept them in the range [8,15,20,30,40,50].”
BackgroundDeepSoCS: A Neural Scheduler for Heterogeneous System-on-Chip (SoC) Resource Scheduling
Sung, T. T., Ha, J., Kim, J., Yahja, A., Sohn, C.-B., & Ryu, B. (2020) · Electronics · doi:10.3390/electronics9060936 · Cited by 19
“e) is more complex to solve and lead to a bad evaluation. That being said, it is ideal for taking a cascade problem, so we train the agent via curriculum learning by gradually decreasing scale values [19]. 3.3. Delayed Consequences The delayed consequence is one of the fundamental challenges in RL [11,20], and often a…”
BackgroundInfluentialGradual Machine Learning for Entity Resolution
Hou, B., Chen, Q., Wang, Y., Nafa, Y., & Li, Z. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3006142 · Cited by 18
Local-Source Enhanced Residual Network for Steganalysis of Digital Images
Ahn, W., Jang, H., Nam, S.-H., Yu, I.-J., & Lee, H.-K. (2020) · IEEE Access · doi:10.1109/access.2020.3011752 · Cited by 14
“3), we build the detector via curriculum learning [40] following previous work [16], described symbolically as 0.”
MethodBatch Prioritization in Multigoal Reinforcement Learning
Vecchietti, L. F., Kim, T., Choi, K., Hong, J., & Har, D. (2020) · IEEE Access · doi:10.1109/access.2020.3012204 · Cited by 14
“This constraint is not feasible in tasks in which a certain type of curriculum learning [17] is needed.”
BackgroundA semi-supervised deep convolutional framework for signet ring cell detection
Ying, H., Song, Q., Chen, J., Liang, T., Gu, J., Zhuang, F., Chen, D. Z., & Wu, J. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.05.119 · Cited by 15
Generalization in Machine Learning via Analytical Learning TheoryPreprint
Kawaguchi, K., Bengio, Y., Verma, V., & Kaelbling, L. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.07426 · Cited by 23
“…how interpolating between the representations of two images (in representation space) corresponds (when projected in image space) to other images that are plausible (are on or near the manifold of natural images), rather than to the simple addition of two natural images (Bengio et al., 2009b).”
BackgroundSelf-citationLearning Anytime Predictions in Neural Networks via Adaptive Loss BalancingPreprint
Hu, H., Dey, D., Hebert, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.06832 · Cited by 14
Interaction-aware Decision Making with Adaptive Strategies under Merging ScenariosPreprint
Hu, Y., Nakhaei, A., Tomizuka, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.06025 · Cited by 14
“Therefore, we introduced a two-stage training process which utilized the curriculum learning [17] strategy.”
MethodLipReading with 3D-2D-CNN BLSTM-HMM and word-CTC modelsPreprint
Margam, D. K., Aralikatti, R. C., Sharma, T., Thanda, A., Pujitha, A. K., Roy, S., & Venkatesan, S. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.12170 · Cited by 27
“Curriculum learning has been shown to provide better performance and faster convergence[17].”
EdgeNet: Semantic Scene Completion from RGB-D images.Preprint
Dourado, A., de Campos, T. E., Kim, H., & Hilton, A. (2019) · arXiv (Cornell University) · Cited by 11
Self-supervised 6D Object Pose Estimation for Robot ManipulationPreprint
Deng, X., Yu, X. J., Mousavian, A., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10159 · Cited by 14
“idea [39] to bootstrap the system sequentially.”
Espresso: A Fast End-to-end Neural Speech Recognition ToolkitPreprint
Wang, Y., Chen, T., Xu, H., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.08723 · Cited by 14
“Curriculum learning [50], which is quite helpful to stabilize training with long sequences (e.”
BackgroundGe, W., Guo, S., Huang, W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.02624 · Cited by 14
“In this work, we propose a weakly-supervised learning approach for instance segmentation (Label-PEnet), which is inspired by the divide-and-conquer idea in curriculum learning [3].”
BackgroundMethodEvolutionary Population Curriculum for Scaling Multi-Agent Reinforcement LearningPreprint
Long, Q., Zhou, Z., Gupta, A., Fang, F., Wu, Y., & Wang, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.10423 · Cited by 137
“It has been extended to deep neural networks on both vision and language tasks (Bengio et al., 2009) and much beyond: Karras et al.”
Curriculum Pre-training for End-to-End Speech TranslationPreprint
Wang, C., Wu, Y., Liu, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.10093 · Cited by 14
“This idea is inspired by the human learning process and is first applied in the context of machine learning by Bengio et al. (2009).”
BackgroundMethodDisentangling User Interest and Popularity Bias for Recommendation with Causal Embedding.Preprint
Zheng, Y., Gao, C., Li, X., He, X., Li, Y., & Jin, D. (2020) · arXiv (Cornell University) · Cited by 14
SWIPENET: Object detection in noisy underwater imagesPreprint
Chen, L., Zhou, F., Wang, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.10006 · Cited by 14
“On the four datasets, Curriculum decays the performance in the 2nd iteration but boosts the performance in the 3rd iteration.”
BackgroundMethodInfluentialText2Event: Controllable Sequence-to-Structure Generation for End-to-end Event ExtractionPreprint
Lu, Y., Lin, H., Xu, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09232 · Cited by 14
“To address the above challenges, we employ a curriculum learning (Bengio et al., 2009; Xu et al., 2020) strategy.”
MethodVázquez-Lema, D., Mosqueira-Rey, E., Hernández-Pereira, E., Fernandez-Lozano, C., Seara-Romera, F., & Pombo-Otero, J. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10799-7 · Cited by 18
“Humans can also play a role in processes such as Curriculum Learning (CL) [29], where a scoring function organizes the dataset in terms of increasing complexity to take advantage of previously learned concepts and to facilitate the abstraction of new concepts.”
Xin, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-22752-2 · Cited by 17
“The training strategy incorporates curriculum learning principles that gradually increase data complexity and introduce additional modalities during the learning process 42 .”
InfluentialOptical Flow Requires Multiple Strategies (but Only One Network)
Schuster, T., Wolf, L., & Gadot, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.732 · Cited by 16
“Curriculum learning [6], inspired by the learning process of humans, was the first method to manipulate the order of samples shown to the model during training.”
MethodDecoupling Inherent Risk and Early Cancer Signs in Image-Based Breast Cancer Risk Models
Liu, Y., Azizpour, H., Strand, F., & Smith, K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59725-2_23 · Cited by 22
“Networks are known to converge faster with easier examples [20, 21], and if it focuses too strongly on increasing confidence of the easy samples [22] learning on the harder long-term risk problem may be crippled.”
BackgroundGupta, K., Mukherjee, D., & Najjaran, H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00934-9 · Cited by 20
“Advancements and applications discussed in “Applications” tackle these challenges to a great extent and can be extended with the use of Curriculum Learning [9].”
BackgroundInfluentialGraph Alignment with Noisy Supervision
Pei, S., Lu, Y., Yu, G., & Zhang, X. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512089 · Cited by 19
“With the development of deep learning and its important role as a powerful learning paradigm in many applications, curriculum learning [5] that controls the order by which training examples are fed to neural networks during training is gaining increased attention [19].”
BackgroundInaguma, H., Dalmia, S., Yan, B., & Watanabe, S. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9687894 · Cited by 14
“Therefore, we reason that CER thresholding is a type of curriculum learning [68] because CTC predictions are used gradually as training progresses.”
BackgroundUncertainty-Aware Graph-Guided Weakly Supervised Object Detection
Zhu, Y., Zhang, Y., Ding, M., & Zuo, W. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3232487 · Cited by 15
Online Domain Adaptation for Semantic Segmentation in Ever-Changing Conditions
Panagiotakopoulos, T., Dovesi, P. L., Härenstam-Nielsen, L., & Poggi, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19830-4_8 · Cited by 29
A Robust Fault Detection Method Based on Neighborhood Preserving Embedding
Sha, X., Luo, C., & Diao, N. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3218557 · Cited by 13
“Bengio et al. [31] first introduced the concept of curriculum learning to alleviate the nonconvex problem.”
BackgroundGraph Attention-Based Curriculum Learning for Mental Healthcare Classification
Ahmed, U., Lin, J. C., & Srivastava, G. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3274486 · Cited by 18
“Curriculum learning is an active learning strategy in which samples are selected based on their level of difficulty, determined by their relevance to the task [35].”
BackgroundLiu, Z. C., Cheng, Y., & Tamura, S. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3281466 · Cited by 19
“1) Cl: Using curriculum in the context of machine learning is first proposed by [25], before [26] coin the term curriculum learning and trains deep neural network models gradually from easier to harder data within a dataset.”
BackgroundCurricular Object Manipulation in LiDAR-based Object Detection
Zhu, Z., Meng, Q., Wang, X., Wang, K., Yan, L., & Yang, J. Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00115 · Cited by 27
“Curriculum learning draws inspiration from the human cognitive process, which begins with easier concepts and gradually moves on to more complicated ones [1, 44].”
BackgroundFederated Multi-Phase Curriculum Learning to Synchronously Correlate User Heterogeneity
Wang, M., Guo, J., & Jia, W. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3307664 · Cited by 16
“CL can accelerate the training of machine learning models, reduce the number of iterations, and achieve a better local optimal state [21].”
BackgroundA Taxonomic Survey of Model Extraction Attacks
Genç, D., Özuysal, M., & Tomur, E. (2023) · IEEE International Conference on Cyber Security and Resilience (CSR) · doi:10.1109/csr57506.2023.10224959 · Cited by 20
“Another possible direction is to increase query efficiency by exploiting advanced training approaches such as curriculum learning [24] and contrastive learning [25].”
BackgroundJiang, J., Kong, D., Hou, K., Huang, X., Zhuang, H., & Fang, Z. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3278097 · Cited by 20
“In this article, we use a hybrid reinforcement training method based on curriculum learning [54], [55], [56] to train the actor-critic networks.”
BackgroundMethodNeural Improvement Heuristics for Graph Combinatorial Optimization Problems
Garmendia, A. I., Ceberio, J., & Mendiburu, A. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3314375 · Cited by 20
“2) Automated Curriculum Learning: Curriculum Learning is a training strategy that involves controlling the difficulty of the samples throughout the process, where the difficulty level is manually increased [37].”
Perturbed Progressive Learning for Semisupervised Defect Segmentation
Wu, Y., Xing, M., Zhang, Y., Xie, Y., Wu, Z., & Qu, Y. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3324188 · Cited by 17
“Inspired by curriculum learning [68], which imitates the human learning procedure to learn from easy to hard, we establish an SIR scheme to divide the unlabeled data into the easy and hard groups and generate pseudo-labels progressively.”
MethodDeep Multitask Learning with Progressive Parameter Sharing
Shi, H., Ren, S., Zhang, T., & Pan, S. J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01824 · Cited by 18
“To achieve that, we propose a dynamic resource allocation strategy at the neuron level by parameterizing the task-specific sharing probability of each neuron of the neural network, conditioning different tasks to be learned simultaneously.”
BackgroundMethodMulti-view fusion for instruction mining of large language model
Huang, H., Xu, B., Liang, X., Chen, K., Yang, M., Zhao, T., & Zhu, C. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102480 · Cited by 21
Deep learning model for differentiating nasal cavity masses based on nasal endoscopy images
Tai, J., Han, M., Choi, B. Y., Kang, S. H., Kim, H., Kwak, J., Lee, D., Lee, T. H., Cho, Y. W., & Kim, T. H. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02517-z · Cited by 16
“Curriculum learning [15], involving the step-by-step training of more sophisticated concepts, could partially solve these complex challenges.”
BackgroundHabib, M. A., Wadud, M. A. H., Patwary, M. F. K., Rahman, M. M., Mridha, M. F., Okuyama, Y., & Shin, J. (2024) · IEEE Access · doi:10.1109/access.2024.3435541 · Cited by 27
Fiorini, L., Bossi, F., & Gruttola, F. D. (2024) · Scientific Reports · doi:10.1038/s41598-024-75263-x · Cited by 20
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.