Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 2,501–2,600 of 5,240
Jayasimhan, A., & Pabitha, P. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112719 · Cited by 6
Chen, Z., Wang, X., Shen, L., et al. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3511515 · Cited by 6
“MVSNet þþ [32] combined attention with a curriculum learning strategy [33] to re fi ne depth estimates, improving the accuracy of foreground depth predictions.”
A Survey of the Application of Neural Networks to Event Extraction
Xie, J., Zhang, Y., Kou, H., et al. (2024) · Tsinghua Science & Technology · doi:10.26599/tst.2023.9010139 · Cited by 6
Two‐View Image Semantic Cooperative Nonorthogonal Transmission in Distributed Edge Networks
Wang, W., Cai, D., Dong, Z., Yu, L., Xu, Y., & Liu, Z. (2024) · International Journal of Intelligent Systems · doi:10.1155/int/5081017 · Cited by 7
Yang, Y., Ge, F., Fan, J., Zhao, J., & Dong, Z. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3516089 · Cited by 8
“Addressing the inherent non-convexity of optimization problems, we draw inspiration from previous studies [52] to devise a three-stage learning paradigm.”
Temporal-Noise-Aware Neural Networks for Suicidal Ideation Prediction Using Physiological Data
Liu, N., Liu, F., Du, X., Shu, Y., Liu, X., Wang, L., Zhao, G., Mu, W., & Liu, Y. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3523928 · Cited by 6
On the Effectiveness of Enrollment Speech Augmentation For Target Speaker Extraction
Li, J., Zhang, K., Wang, S., Li, H., Mak, M., & Lee, K. A. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832217 · Cited by 14
“The ‘CL’ strategy, which implements curriculum learning [47, 48] by first training without augmentations and then with them, proves effective for improving performance across most evaluation metrics.”
SoftTex: Soft Robotic Arm With Learning-Based Textile Proprioception
Pagliarani, N., Alessi, C., Arleo, L., Campinoti, G., Maselli, M., Falotico, E., & Cianchetti, M. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3546070 · Cited by 12
“To mitigate the hysteresis effects and improve generalization, we employ a curriculum learning strategy[25].”
SHADE-AD: An LLM-Based Framework for Synthesizing Activity Data of Alzheimer’s Patients
Fu, H., Chen, H., Lin, S., & Xing, G. (2025) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3715014.3722062 · Cited by 13
How Can Everyday Users Efficiently Teach Robots by Demonstration?
Sakr, M., Zhang, Z., Li, B., Zhang, H., der Loos, H. F. M. V., Kulić, D., & Croft, E. A. (2025) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3737892 · Cited by 18
“…the “teaching dimension model”, which represents the minimum number of instances a teacher must reveal to uniquely identify any target concept [25], and the “curriculum learning principle”, which presents training examples in a structured sequence, progressing from simple to complex concepts [3].”
Group Norm for Learning Structured SVMs with Unstructured Latent Variables
Chen, D., Batra, D., & Freeman, W. T. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.58 · Cited by 7
“Our experiments on digit recognition and object detection show that our approach is indeed able to control the complexity of latent space without any significant loss in accuracy of the learnt model.”
BackgroundSelf-Paced Learning Based Multi-view Spectral Clustering
Yu, H. Z., Lian, Y., Zong, L., et al. (2017) · Proceedings - International Conference on Tools with Artificial Intelligence, TAI · doi:10.1109/ictai.2017.00013 · Cited by 6
“However, in the curriculum learning, it doesn’t give the readily computable measure of the easiness of samples(in our paper we just measure the easiness of the view) [20].”
BackgroundHow Clever Is the FiLM Model, and How Clever Can it Be?
Kuhnle, A., Xie, H., & Copestake, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11018-5_15 · Cited by 6
“We investigate how two approaches – broader training sets including simpler instances, and a version of curriculum learning (Elman, 1993; Bengio et al., 2009) – can make the difference between no learning at all and perfectly solving these datasets.”
BackgroundMethodLearning Driving Behaviors for Automated Cars in Unstructured Environments
Kaushik, M., & Krishna, K. M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11021-5_36 · Cited by 6
Multi-task Learning for Target-Dependent Sentiment Classification
Gupta, D., Singh, K., Chakrabarti, S., & Chakraborty, T. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-16148-4_15 · Cited by 7
“From the perspective of curriculum learning [2], this seems suboptimal: representation borrowed from ar X iv :1 90 2.”
BackgroundYu, S.-I., Yang, Y., Xu, Z., Xu, S., Meng, D., Mao, Z., Ma, Z., Lin, M., Li, X., Li, H., et al. (2016) · ITE Transactions on Media Technology and Applications · doi:10.3169/mta.4.227 · Cited by 5
Curriculum Learning in Sentiment Analysis
Sido, J., & Konopík, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-26061-3_45 · Cited by 6
Liu, K.-C., Shen, Y., Klopp, J. P., & Chen, L. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00744 · Cited by 7
“[44] applies domain adaptation in a curriculum learning [4] fashion, learning scene parsing from tasks that are less sensitive to the aforementioned domain discrepancy.”
BackgroundCumulative Training and Transfer Learning for Multi-Robots Collision-Free Navigation Problems
Nguyen, T. T., Hatua, A., & Sung, A. H. (2019) · IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) · doi:10.1109/uemcon47517.2019.8992945 · Cited by 6
“Our proposed paradigm is developed from the idea of curriculum training [25] which requires the scenarios to be introduced to the robot agents from easiest one to the most complex ones such as to in Fig.”
MethodLow-budget label query through domain alignment enforcement
Saltori, C., Rota, P., Sebe, N., & Almeida, J. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103485 · Cited by 7
“[3] introduced the concept of curriculum learning in which a model is trained starting from the easiest samples and then is gradually introduced with more complex samples along the training.”
BackgroundImproving Singing Voice Separation Using Curriculum Learning on Recurrent Neural Networks
Kang, S., Park, J., & Jang, G. (2020) · Applied Sciences · doi:10.3390/app10072465 · Cited by 6
“To overcome these limitations, we propose a new single-channel singing voice separation approach based on curriculum learning [11].”
BackgroundMethodA learning approach with incomplete pixel-level labels for deep neural networks
Nguyen, N.-V., Rigaud, C., Revel, A., & Burie, J.-C. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.06.025 · Cited by 6
Pan, X., Wei, D., Zhao, Y., Ma, M., & Wang, H. (2020) · IEEE Access · doi:10.1109/access.2020.3047884 · Cited by 7
“Most of the optimization functions used in deep learning are nonconvex functions [4], which increase the difficulty of finding global minima.”
BackgroundStreaming End-to-End Speech Recognition with Jointly Trained Neural Feature Enhancement
Kim, C., Garg, A., Gowda, D., Mun, S., & Han, C. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414117 · Cited by 7
“Inspired by Curriculum Learning [1], we propose two training strategies in this work: Gradual Application of Enhanced Feature (GAEF) and Gradual Reduction of Enhancement Loss (GREL).”
BackgroundMethodInfluentialAdaptive Sample Selection for Robust Learning under Label Noise
Patel, D., & SASTRY, P. S. (2021) · NOT FOUND REPOSITORY (Indian Institute of Science Bangalore) · doi:10.48550/arxiv.2106.15292 · Cited by 6
“Curriculum learning (CL) as proposed in [4] is the designing of an (optimal) manner of sequencing of training samples (based on a notion of easiness of an example) to improve the performance of the learning system.”
BackgroundLearning to Inference with Early Exit in the Progressive Speech Enhancement
Li, A., Zheng, C., Zhang, L., et al. (2021) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco54536.2021.9616248 · Cited by 6
“Motivated by curriculum learning concept [8], progressive learning (PL) based SE algorithms begin to thrive recently [9]– [11], which have been demonstrated to exhibit better performance over the conventional “black-box” DNN framework.”
BackgroundEdit Distance Based Curriculum Learning for Paraphrase Generation
Kadotani, S., Kajiwara, T., Arase, Y., & Onizuka, M. (2021) · doi:10.18653/v1/2021.acl-srw.24 · Cited by 7
Elastic Knowledge Distillation by Learning From Recollection
Fu, Y., Li, S., Zhao, H., Wang, W., Fang, W., Zhuang, Y., Pan, Z., & Li, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3107317 · Cited by 6
“Inspired by curriculum learning [6], we claim that a model should start training with this relatively easier task (learn more from moderately confident recollection) and gradually increase the weight of the difficult one (a highly confident recollection) to further refine the prediction.”
BackgroundMethodInfluentialGenerating Diverse 3D Reconstructions from a Single Occluded Face Image
Dey, R., & Boddeti, V. N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00160 · Cited by 8
“We adopt a curriculum learning [2] approach to overcome this challenge and progressively introduce larger occlusions during the training process, i.”
MethodDo, S., & Lee, C. (2021) · International Conference on Control, Automation and Systems (ICCAS) · doi:10.23919/iccas52745.2021.9649915 · Cited by 6
Glance: A Generative Approach to Interactive Visualization of Voluminous Satellite Imagery
Mitra, S., Rammer, D., Pallickara, S., et al. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671537 · Cited by 6
“The mapping between ( I LR + , g ) to the true residual for I SR becomes increasingly more complicated as the upscale [27], where more complex upscaling models are built on top of trained models that handle relatively simpler task of a lower upscaling factor.”
BackgroundImproving Reverberant Speech Separation with Synthetic Room Impulse Responses
Aralikatti, R. C., Ratnarajah, A. J., Tang, Z., & Manocha, D. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688203 · Cited by 10
Faster, Better Blink Detection through Curriculum Learning by Augmentation
Al‐Hindawi, A., Vizcaychipi, M. P., & Demiris, Y. (2022) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3517031.3529617 · Cited by 6
“Similarly, curriculum learning has been demonstrated to increase the performance of classification [Bengio et al. 2009].”
BackgroundSuzuki, S., Takeda, S., Makishima, N., Ando, A., Masumura, R., & Shouno, H. (2022) · IEEE Access · doi:10.1109/access.2022.3186101 · Cited by 9
“Sinha it et al. [10] determined the strength of blur filters in a curriculum manner [26] and demonstrated that they work well for many image recognition tasks.”
BackgroundHuerta, E. A., & Zhao, Z. (2022) · Springer eBooks · doi:10.1007/978-981-16-4306-4_47 · Cited by 7
A Faster Approach For Direct Speech to Speech Translation
Shankarappa, R. T., & Tiwari, S. (2022) · doi:10.1109/wintechcon55229.2022.9832314 · Cited by 6
A new self-paced learning method for privilege-based positive and unlabeled learning
Liu, B., Liu, J., Xiao, Y., Chen, Q., Wang, K., Huang, R., & Li, L. (2022) · Information Sciences · doi:10.1016/j.ins.2022.07.143 · Cited by 7
Cross-Domain Cross-Set Few-Shot Learning via Learning Compact and Aligned Representations
Chen, W., Zhang, Z., Wang, W., Wang, L., Wang, Z., & Tan, T. N. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20044-1_22 · Cited by 13
McMahon, T., Sivaramakrishnan, A., Kedia, K. S., et al. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9982136 · Cited by 6
“To minimize the data requirements for training the controller, an explicit curriculum learning approach [32] is explored in this work, where the controller is first pre-trained in a flat environment and then trained in the sloped environments.”
MethodLength-Based Curriculum Learning for Efficient Pre-training of Language Models
Nagatsuka, K., Broni-Bediako, C., & Atsumi, M. (2022) · New Generation Computing · doi:10.1007/s00354-022-00198-8 · Cited by 23
“Furthermore, the pacing functions are compared with the Baby Steps [16, 23], which is often used as a basic training scheduler as described in Sect.”
MethodInfluentialSensor-Based Navigation Using Hierarchical Reinforcement Learning
Gebauer, C., Dengler, N., & Bennewitz, M. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-22216-0_37 · Cited by 6
“For training, we apply curriculum learning [3] and increase the difficulty step by step, i.”
PLMCL: Partial-Label Momentum Curriculum Learning for Multi-label Image Classification
Abdelfattah, R., Zhang, X., Wu, Z., Wu, X., Wang, X., & Wang, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25063-7_3 · Cited by 10
LSCALE: Latent Space Clustering-Based Active Learning for Node Classification
Liu, J., Wang, Y., Hooi, B., et al. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26387-3_4 · Cited by 6
“Inspired by curriculum learning approaches [2], we set an exponentially decaying weight as follows: α = λ|L t|, (8) where |L| is the number of labelled nodes at step t.”
MethodLearning to Transfer In‐Hand Manipulations Using a Greedy Shape Curriculum
Zhang, Y., Clegg, A., Ha, S., Turk, G., & Ye, Y. (2023) · Computer Graphics Forum · doi:10.1111/cgf.14741 · Cited by 11
“Curriculum learning [BLCW09] is one of the common techniques for solving challenging problems.”
BackgroundSelf-Supervised Video Representation Learning by Serial Restoration With Elastic Complexity
Chen, Z., Wang, H., & Chen, C. W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3293727 · Cited by 7
“Curriculum learning is first proposed in [39] by analogy with humans that learn knowledge from easy to hard.”
BackgroundMethodEasy-to-Hard Learning for Information Extraction
Gao, C., Zhang, W., Lam, W. P., & Bing, L. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.754 · Cited by 13
Learning Multiple-Gait Quadrupedal Locomotion via Hierarchical Reinforcement Learning
Wei, L., Li, Y., Ai, Y., Wu, Y., Xu, H., Wang, W., & Hu, G. (2023) · International Journal of Precision Engineering and Manufacturing · doi:10.1007/s12541-023-00885-6 · Cited by 6
“In prior work, curriculum learning has shown its potential in terms of helping RL algorithms to get higher performance [14, 29].”
BackgroundA Δ-learning strategy for interpretation of spectroscopic observables
Watson, L., Pope, T. J., Jay, R. M., Banerjee, A., Wernet, P., & Penfold, T. J. (2023) · Structural Dynamics · doi:10.1063/4.0000215 · Cited by 7
Dynamic semantic structure distillation for low-resolution fine-grained recognition
Liang, M., Huang, S., & Liu, W. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.110216 · Cited by 10
Inversion dynamics of class manifolds in deep learning reveals tradeoffs underlying generalization
Ciceri, S., Cassani, L., Osella, M., Rotondo, P., Valle, F., & Gherardi, M. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-023-00772-9 · Cited by 14
“…of stragglers may lend itself to applications as a dataset-distillation aid, for the design of dataset-specific curriculum learning protocols [39], and it can inspire, in the field of explainable AI, post-hoc methods for the selection of exceptionally influential instances within the data set…”
MethodSuzuki, S., Yamaguchi, S., Takeda, S., Kanai, S., Makishima, N., Ando, A., & Masumura, R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00405 · Cited by 16
“[52] attempted to mitigate the tradeoff by using a curriculum learning strategy [2].”
BackgroundStructure-Informed Positional Encoding for Music Generation
Agarwal, M., Wang, C., & Richard, G. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448149 · Cited by 10
“We use a 2-layer Transformer decoder with 4 heads, trained with a learning rate scheduler, early stopping and curriculum learning [19] on the sequence length.”
MethodFuruya, T., Chen, Z., Ohbuchi, R., & Kuang, Z. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104025 · Cited by 7
“We speculate that self-distillation facilitates convergence to a better solution, which is one of the benefits of curriculum learning [68], resulting in accurate 3D point set features.”
BackgroundUnsupervised Layer-Wise Model Selection in Deep Neural Networks
Arnold, L., Hélène, P.-M., & Michèle, S. (2010) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-60750-606-5-915 · Cited by 6
“Along the same lines, the choice of the examples (curriculum learning [5]) used to train the RBM, will be investigated w.r.t. the unsupervised quality of the hidden units: The challenge would be to define an intrinsic, unsupervised, measure to order the examples and construct a curriculum.”
K-component recurrent neural network language models using curriculum learning
Shi, Y., Larson, M., & Jonker, C. M. (2013) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2013.6707696 · Cited by 6
“The curriculum learning strategy was recently revisited by [3] and [14].”
BackgroundMethodBootstrapping Dependency Grammar Inducers from Incomplete Sentence Fragments via Austere Models
Spitkovsky, V. I., Alshawi, H., & Jurafsky, D. S. (2012) · International Colloquium on Grammatical Inference · Cited by 5
On the Recursive Neural Networks for Relation Extraction and Entity Recognition
Khashabi, D. (2013) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 10
Multi-Paced Dictionary Learning for cross-domain retrieval and recognition
Xu, D., Song, J., Alameda-Pineda, X., Ricci, E., & Sebe, N. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7900132 · Cited by 5
“Curriculum [6] and self-paced [4] learning develop from the idea that models must be learned in an incremental fashion, using the easy samples before the difficult ones.”
BackgroundSequence-to-sequence models for trajectory deformation of dynamic manipulation
Kutsuzawa, K., Sakaino, S., & Tsuji, T. (2017) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon.2017.8216904 · Cited by 6
“To deal with the issue, this paper proposes a training method based on curriculum learning [20].”
MethodKocmi, T., Sudarikov, R., & Bojar, O. (2018) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w18-6416 · Cited by 10
“Similar to transfer learning is also curriculum learning (Bengio et al., 2009; Kocmi and Bojar, 2017), where the training data are ordered from foreign out-of-domain to the in-domain training examples.”
BackgroundUnrestricted Random Sampling of data batch to improve the efficiency of neural networks
Korchi, A. E., & Ghanou, Y. (2019) · Conference of the Moroccan Classification Society: New Challenges in Data Sciences · doi:10.1145/3314074.3314102 · Cited by 6
Robust boosting via self-sampling
Liu, X., Luo, S., & Pan, L. (2019) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.105424 · Cited by 6
A Method for Quantitative Steganalysis Based on Deep Learning
Sun, Y., & Li, T. (2019) · International Conference on Information Systems and Computer Aided Education (ICISCAE) · doi:10.1109/iciscae48440.2019.221640 · Cited by 6
“In view of the problem that the detection difficulty varies under different embedding rates, the curriculum learning strategy [11] adopted in YeNet is followed, and we set about from the tasks with a high embedding rate and a low detection difficulty.”
BackgroundEmpowering Active Learning to Jointly Optimize System and User Demands
Lee, J.-U., Meyer, C. M., & Gurevych, I. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.390 · Cited by 14
“of the user-oriented sampling strategy to curriculum learning (Bengio et al., 2009), which opts to organize model training in a meaningful way.”
BackgroundHierarchical Class-Based Curriculum Loss
Goyal, P., Choudhary, D., & Ghosh, S. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/337 · Cited by 11
Evolving models for incrementally learning emerging activities
Ye, J., & Callus, E. (2020) · Journal of Ambient Intelligence and Smart Environments · doi:10.3233/ais-200566 · Cited by 6
“Ideally, the model can gain higher performance if it trains with easier classes and then gradually learns classes with increasing difficulty level [5,11].”
Robust sparse coding via self-paced learning for data representation
Feng, X., & Wu, S. (2020) · Information Sciences · doi:10.1016/j.ins.2020.08.097 · Cited by 7
Evolutionary training and abstraction yields algorithmic generalization of neural computers
Tanneberg, D., Rueckert, E., & Peters, J. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00255-1 · Cited by 7
Duan, S., Xiong, J., & Qian, H. (2020) · International Journal of Structural Stability and Dynamics · doi:10.1142/s0219455421500474 · Cited by 6
When Deep Classifiers Agree: Analyzing Correlations Between Learning Order and Image Statistics
Pliushch, I., Mundt, M., Lupp, N., & Ramesh, V. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20074-8_23 · Cited by 18
“Further, as the next step, these insights could drive us towards the design of a learning curriculum [29,27,4,16], in which the dataset instances could be sampled according to a high correlation between observed agreement and the inspected metrics.”
BackgroundTHE ROLE OF SEQUENCES FOR INCREMENTAL LEARNING
Wenzel, S., & Hotz, L. (2010) · International Conference on Agents and Artificial Intelligence (ICAART) · doi:10.5220/0002762604340439 · Cited by 6
“Recently, the term curriculum learning was used by (Bengio et al., 2009) for finding sequences for training neural networks.”
Self-paced Safe Co-training for Regression
Min, F., Li, Y., & Liu, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05936-0_6 · Cited by 6
Ecosystem Models Based on Artificial Intelligence
Strannegård, C., Engsner, N., Eisfeldt, J., Endler, J. A., Hansson, A., Lindgren, R., Mostad, P., Olsson, S., Perini, I., Reese, H., et al. (2022) · Swedish Artificial Intelligence Society Workshop (SAIS) · doi:10.1109/sais55783.2022.9833026 · Cited by 8
Ran, L., Ji, C., Zhang, S., Zhang, X., & Zhang, Y. (2022) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc55077.2022.9886060 · Cited by 7
“Bengio [33] first proposed the concept of Curriculum learning [34], which is a training strategy that imitates the learning process of human beings and advocates that the model should start learning from easy samples and gradually advance to complex samples and knowledge.”
BackgroundDukkipati, A., Banerjee, R., Ayyagari, R. S., & Parmar, D. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981607 · Cited by 9
“This method shares a striking resemblance with curriculum-based learning approaches [8], but here the curriculum naturally arises from the necessity of traversing the environment.”
MethodResultCMW-Net: an adaptive robust algorithm for sample selection and label correction
Shu, J., Yuan, X., & Meng, D. (2023) · National Science Review · doi:10.1093/nsr/nwad084 · Cited by 6
“In our implementation, we automatically generate a certain number of the most confident training samples in each iteration of the training process as meta-data, which is soundly motivated by the curriculum learning methodology [6,7], use a common semi-supervised learning technique [8] to produce pseudo label z and emp…”
MethodCurriculum learning for printed text line recognition of ligature-based scripts
Ul-Hasan, A., Shafaity, F., & Liwicki, M. (2015) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2015.7333912 · Cited by 6
Asynchronous Data Aggregation for Training End to End Visual Control Networks
Monfort, M., Johnson, M., Oliva, A., & Hofmann, K. (2017) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/3091125.3091204 · Cited by 6
SPGLAD: A Self-paced Learning-Based Crowdsourcing Classification Model
Zhang, X., Shi, H., Li, Y., & Liang, W. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67274-8_17 · Cited by 6
SEE: Towards Semi-Supervised End-to-End Scene Text Recognition
Bartz, C., Yang, H., & Meinel, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12242 · Cited by 26
“nd that, when trained from scratch, a network that shall detect and recognize more than two text lines does not converge. In order to overcome this problem we designed a curriculum learning strategy (Bengio et al. 2009) for training the system. The complexity of the supplied training images under this curriculum is gr…”
BackgroundMethodPubSE: A Hierarchical Model for Publication Extraction from Academic Homepages
Zhang, Y., Qi, J., Zhang, R., et al. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1123 · Cited by 6
“Alternating training method : Inspired by the training procedure of curriculum learning (Bengio et al., 2009) and soft-landing (He et al., 2016), we adopt an alternating training procedure controlled by the following function: where T controls the period of the function, and k is the number of…”
MethodEdge-Nodes Representation Neural Machine for Link Prediction
Xu, G., Wang, X., Wang, Y., et al. (2019) · Algorithms · doi:10.3390/a12010012 · Cited by 6
“And we adopt the Adam update rule [36] for optimization with a learning rate of 0.”
MethodHu, S., Peng, J., Fu, Y., & Li, L. (2019) · Remote Sensing · doi:10.3390/rs11091114 · Cited by 6
“To simultaneously improve the joint representation and linear representation abilities of the JSR model, we adopt a self-paced learning (SPL) strategy [27–29] to select feature-neighboring pixels and propose a self-paced KJSR (SPKJSR) model.”
Collaborative learning with corrupted labels
Wang, Y., Huang, R., Huang, G. M., Song, S., & Wu, C. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.02.010 · Cited by 7
Curriculum Learning for Face Recognition
Buyuktas, B., Erdem, Ç. E., & Erdem, A. T. (2020) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco47968.2020.9287639 · Cited by 11
“It is used mainly to improve the speed of convergence and achieve better generalization by increasing the complexity of the training data gradually [21].”
MethodRLTS: Robust Learning Time-Series Shapelets
Yamaguchi, A., Maya, S., & Ueno, K. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67658-2_34 · Cited by 7
“Unlike in conventional curriculum learning [4], learners dynamically generate their own curriculum according to what they have already learned.”
BackgroundCoinbot: Intelligent Robotic Coin Bag Manipulation Using Artificial Brain
Gonnochenko, A., Semochkin, A., Egorov, D. E., Statovoy, D., Zabihifar, S., Postnikov, A., Seliverstova, E., Zaidi, A. M., Stemmler, J., & Limkrailassiri, K. (2021) · International Conference on Automation, Robotics and Applications · doi:10.1109/icara51699.2021.9376455 · Cited by 6
“Second, following the methodology of curriculum learning [18], [19], we guide the training of our models by progressively increasing the complexity of the training environment with the agent’s performance.”
MethodGraph-based reasoning attention pooling with curriculum design for content-based image retrieval
Zhu, X., Wang, H., Liu, P., Yang, Z., & Qian, J. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2021.104289 · Cited by 8
Learning Size and Fit from Fashion Images
Karessli, N., Guigourès, R., & Shirvany, R. (2020) · Lecture notes in social networks · doi:10.1007/978-3-030-55218-3_6 · Cited by 6
Dynamic-structured Semantic Propagation NetworkPreprint
Liang, X., Zhou, H., & Xing, E. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.06067 · Cited by 6
“In the spirit of curriculum learning [2] that gradually increases the target difficulty levels and exploits previously learned knowledge for learning new fine-grained”
BackgroundDeepLogic: End-to-End Logical Reasoning.Preprint
Cingillioglu, N., & Russo, A. (2018) · arXiv (Cornell University) · Cited by 6
InstaNAS: Instance-aware Neural Architecture SearchPreprint
Cheng, A.-C., Lin, C. H., Juan, D.-C., Wei, W., & Sun, M. (2018) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.1811.10201 · Cited by 55
“Inspired by curriculum learning [3], we design a dynamic quadratic reward function for RC toward the policy collapsing problem.”
MethodAccelerating Minibatch Stochastic Gradient Descent using Typicality SamplingPreprint
Peng, X., Li, L., & Wang, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.04192 · Cited by 6
How hard can it be? Estimating the difficulty of visual search in an imagePreprint
Ionescu, R. T., Alexe, B., Leordeanu, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08280 · Cited by 6
“Curriculum learning [4] can help to optimize the training of deep learning models.”
BackgroundProbabilistic Label Relation Graphs with Ising ModelsPreprint
Ding, N., Deng, J. H., Murphy, K. P., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.01428 · Cited by 6
“This can be thought of as a form of curriculum learning [2] by training with a simpler model (HEX graph) with exact inference first.”
BackgroundBecoming the Expert - Interactive Multi-Class Machine TeachingPreprint
Johns, E., Aodha, O. M., & Brostow, G. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.07575 · Cited by 6
“Similar to curriculum learning [5], our strategy initially focuses on representative images and then introduces more difficult ones over time, as the student’s performance improves.”
BackgroundTowards Open-Domain Named Entity Recognition via Neural Correction Models.Preprint
Zhu, M., Deng, Z., Xiong, W., Yu, M., Zhang, M., & Wang, W. Y. (2019) · arXiv (Cornell University) · Cited by 6
Augmenting learning using symmetry in a biologically-inspired domainPreprint
Mishra, S., Abdolmaleki, A., Guez, A., Trochim, P., & Precup, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00528 · Cited by 8
“In robotics and continuous control applications, some ways to improve the real-world suitability and sample efficiency of RL agents include curriculum learning [4], learning from motion-capture data [9], and using constraintbased objective functions [5].”
BackgroundLearning locomotion : symmetry and torque limit considerations
Abdolhosseini, F. (2019) · cIRcle (University of British Columbia) · doi:10.14288/1.0383251 · Cited by 7
“This leads us to curriculum learning [3].”
Topology Aware Deep Learning for Wireless Network OptimizationPreprint
Zhang, S., Yin, B., & Cheng, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.08336 · Cited by 6
“We take a curriculum training [51] approach to organize the samples and conduct training.”
Method
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.