Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 1,201–1,300 of 5,240
Self-Paced Ensemble Learning for Speech and Audio Classification
Ristea, N.-C., & Ionescu, R. T. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-155 · Cited by 36
“We underline that SPL can be seen as a particular case of curriculum learning [3, 4], in which the difficulty of the examples is estimated by the model itself, hence the self-pacing.”
BackgroundSyntax-guided text generation via graph neural network
Guo, Q., Qiu, X., Xue, X., & Zhang, Z. (2021) · Science China Information Sciences · doi:10.1007/s11432-019-2740-1 · Cited by 35
Partially Non-Autoregressive Image Captioning
Fei, Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16219 · Cited by 42
“To solve this problem, we propose an annealing dividing strategy with curriculum learning (Bengio et al. 2009).”
MethodReinforced History Backtracking for Conversational Question Answering
Qiu, M., Huang, X., Chen, C., Ji, F., Qu, C., Wei, W., Huang, J., & Zhang⋆, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17617 · Cited by 35
“Recall that our training scheme is inspired by curriculum learning (Bengio et al. 2009), where we first learn the policy from examples with only one history, followed by learning from examples with two history turns, and so on so forth.”
xMoCo: Cross Momentum Contrastive Learning for Open-Domain Question Answering
Yang, N., Wei, F., Jiao, B., Jiang, D., & Yang, L. (2021) · doi:10.18653/v1/2021.acl-long.477 · Cited by 35
“To this end, one possible direction is to employ curriculum learning (Bengio et al., 2009).”
BackgroundCurriculum-Based Meta-learning
Zhang, J., Song, J., Yao, Y., & Gao, L. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475335 · Cited by 29
“In the family of curriculum learning [5, 13], the idea of learning from (cid:2)(cid”
BackgroundMethodInfluentialInterpolative Distillation for Unifying Biased and Debiased Recommendation
Ding, S., Feng, F., He, X., Jin, J., Wang, W., Liao, Y., & Zhang, Y. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3532002 · Cited by 34
“In other words, the student tends to learn the easier aspects of knowledge since the smaller distance makes it easier to follow the corresponding teacher.”
BackgroundClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization
Wang, J., Meng, F., Lu, Z., Zheng, D., Li, Z., Qu, J., & Zhou, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.526 · Cited by 53
“The pseudo-XLDS samples together with the real XLDS samples from XMediaSum40k jointly train end-to-end models via a simple curriculum learning (Bengio et al., 2009) strategy, i.e., the training data is converted from pseudo data to real data in each epoch.”
BackgroundMethodSpatially and Spectrally Consistent Deep Functional Maps
Sun, M., Mao, S., Jiang, P., Ovsjanikov, M., & Huang, R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01333 · Cited by 51
“Based upon the above analysis, we propose a novel up-dating scheme, which is inspired by curriculum learning [5].”
MethodZhang, J., Huang, Z., & Ohn-Bar, E. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00754 · Cited by 39
“Curriculum and Self-paced Learning: Approaches for curriculum learning methods often structure learning from various difficulty samples [5,24,37,39,44,59,61,63,69,84].”
BackgroundChoudhury, M., Bali, K., Sitaram, S., et al. (2017) · International conference natural language processing · Cited by 27
Mobahi, H., Zitnick, C. L., & Ma, Y. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247869 · Cited by 36
“Similar concept for discrete spaces has also been used [4, 5, 21, 38].”
BackgroundTask Loss Estimation for Sequence PredictionPreprint
Bahdanau, D., Serdyuk, D., Brakel, P., Ke, N. R., Chorowski, J., Courville, A. C., & Bengio, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06456 · Cited by 32
“In the context of Encoder-Decoders for sequence generation, a curriculum learning (Bengio et al., 2009) strategy has been proposed to address the discrepancy between the training and testing conditions of models trained with maximum likelihood (Bengio et al., 2015).”
MethodSelf-citationProceedings of the First Workshop on Neural Machine Translation
(2017) · doi:10.18653/v1/w17-32 · Cited by 30
PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum LearningPreprint
Bao, S., Huang, H., Wang, F., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.16779 · Cited by 27
“In this work, we try to scale up PLATO to PLATO-2 and introduce an effective training schema via curriculum learning (Bengio et al., 2009).”
MethodActive Learning: Problem Settings and Recent DevelopmentsPreprint
Hino, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.04225 · Cited by 68
“se of incorrect labels or outliers. Although not stated in this paper, active learning has been applied to screening such samples (Abe et al., 2006). In addition, methods such as curriculum learning (Bengio et al., 2009) and self-paced learning (Kumar et al., 2010), where samples are selected in an appropriate order f…”
MethodVision-force-fused curriculum learning for robotic contact-rich assembly tasks
Jin, P., Lin, Y., Song, Y., Li, T., & Yang, W. P. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2023.1280773 · Cited by 38
BoNuS: Boundary Mining for Nuclei Segmentation With Partial Point Labels
Lin, Y., Wang, Z., Zhang, D., Cheng, K.-T. T., & Chen, H. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3355068 · Cited by 42
“2) Pseudo Label Selection With Curriculum Learning: Drawing inspiration from curriculum learning [42], we train the detection module in a single shot instead of iteratively training it with different confidence thresholds.”
BackgroundCroitoru, F.-A., Ristea, N.-C., Ionescu, R. T., & Sebe, N. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02186-5 · Cited by 47
“[11] as a training strategy that helps machine learning models to generalize better when the training examples are presented in the ascending order of their difficulty.”
InfluentialWu, X., Wu, X., Yang, B., Zhou, L., Guo, C., Qiu, X., Hu, J., Sheng, Z., & Jensen, C. S. (2024) · The VLDB Journal · doi:10.1007/s00778-024-00872-x · Cited by 64
“To facilitate this learning process on all 2 L samples, we employ a data-level curriculum learning approach [4].”
Toward Open-World Remote Sensing Imagery Interpretation: Past, present, and future
Lang, C., Cheng, G., Wu, J., Li, Z., Xie, X., Li, J., & Han, J. (2024) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2024.3505602 · Cited by 35
Belinkov, Y., Lei, T., Barzilay, R., & Globerson, A. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00203 · Cited by 28
“There are various approaches to training vector representations (Collobert and Weston, 2008; Bengio et al., 2009). Here we chose to focus on the Skip-gram method recently proposed by Mikolov et al. (2013a). The Skip-gram model maximizes the average log-probability of every word generating its context, which is modeled…”
BackgroundMethodEnsemble Partitioning for Unsupervised Image Categorization
Dai, D., Prasad, M., Leistner, C., & Gool, L. V. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33712-3_35 · Cited by 25
“Besides technical differences, the main benefit of our work compared to [19] is that we do not rely on pre-selected learning schedules.”
InfluentialSegmentation in Weakly Labeled Videos via a Semantic Ranking and Optical Warping Network
Yang, L., Han, J., Zhang, D., Liu, N., & Zhang, D. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2834221 · Cited by 26
“To reduce the learning difficulty, we introduce the curriculum learning [43], [44] into the network training process.”
MethodSelf-paced multi-task clustering
Ren, Y., Que, X., Yao, D., & Xu, Z. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.03.062 · Cited by 26
“Self-Paced Learning Curriculum learning (CL) is an instructor-driven model that organizes examples in a meaningful order under the premise of prior knowledge [29].”
BackgroundEvolutionary Data Measures: Understanding the Difficulty of Text Classification Tasks
Collins, E., Rozanov, N., & Zhang, B. (2018) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k18-1037 · Cited by 43
“Curriculum learning may also have a similar effect to pre-training (Bengio et al., 2009).”
Semi-supervised hotspot detection with self-paced multi-task learning
Chen, Y., Lin, Y., Gai, T., et al. (2019) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1145/3287624.3287685 · Cited by 26
“As unlabeled samples are gradually introduced into training, the model is essentially training from “easy” to “mature” through each round [28].”
BackgroundSelf-paced learning with privileged information
Xu, W., Liu, W., Chi, H., Qiu, S., & Jin, Y. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.072 · Cited by 25
Learning to Select Base Classes for Few-Shot Classification
Zhou, L., Cui, P., Jia, X., Yang, S., & Tian, Q. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00468 · Cited by 33
“Different from Curriculum Learning aiming to speed up learning of provided classes, we focus on learning to select base classes in a transfer learning manner, where the selected base classes are used for classification on novel classes.”
InfluentialLearning with Weak Supervision for Email Intent Detection
Shu, K., Mukherjee, S., Zheng, G., Awadallah, A. H., Shokouhi, M., & Dumais, S. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401121 · Cited by 52
“This is similar to curriculum learning [5] where a training schedule is used to first learn from easy samples followed by difficult ones.”
BackgroundComputational co-design of fibrous architecture
Menges, A., Kannenberg, F., & Zechmeister, C. (2022) · Architectural Intelligence · doi:10.1007/s44223-022-00004-x · Cited by 26
“Assuming a sufficiently varied training curriculum (Bengio et al., 2009), the virtual agent could work well on a large variety of fibre components.”
BackgroundArgumentation: A calculus for Human-Centric AI
Dietz, E., Kakas, A. C., & Michael, L. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.955579 · Cited by 40
“, 2015), curriculum learning (Bengio et al., 2009) and continual learning (De Lange et al.”
BackgroundMachine Learning Methods in Solving the Boolean Satisfiability Problem
Guo, W., Zhen, H., Li, X., Luo, W., Yuan, M., Jin, Y., & Yan, J. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1396-2 · Cited by 51
“The authors employed the REINFORCE algorithm[79] to optimize the policy network.”
MethodImproving Long-Tail Item Recommendation with Graph Augmentation
Luo, S., Ma, C., Xiao, Y., & Song, L. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3614929 · Cited by 53
“Drawing inspiration from the concept of curriculum learning [2], we propose a novel two-stage training strategy for our proposed method.”
MethodDistributed deep reinforcement learning based on bi-objective framework for multi-robot formation
Li, J., Liu, Q., & Chi, G. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.11.063 · Cited by 28
Probabilistic Label Relation Graphs with Ising Models
Ding, N., Deng, J., Murphy, K. P., & Neven, H. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.138 · Cited by 33
“This can be thought of as a form of curriculum learning [2] by training with a simpler model (HEX graph) with exact inference first.”
BackgroundMultitask Learning of Context-Dependent Targets in Deep Neural Network Acoustic Models
Bell, P., Świętojański, P., & Renals, S. J. (2016) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2016.2630305 · Cited by 31
“This is similar to the motivation for curriculum learning [27], where the entropy of the task is gradually increased as learning progresses.”
ResultTPCKT: Two-Level Progressive Cross-Media Knowledge Transfer
Huang, X., & Peng, Y. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2911456 · Cited by 25
“first learns from easy samples, and then samples with progressively ascending difficulties [38].”
BackgroundAn integrative analysis system of gene expression using self-paced learning and SCAD-Net
Huang, H., & Liang, Y. Q. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.06.016 · Cited by 26
A Customized Convolutional Neural Network with Low Model Complexity for JPEG Steganalysis
Huang, J., Ni, J., Wan, L., & Yan, J. (2019) · ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) · doi:10.1145/3335203.3335734 · Cited by 27
“Therefore, curriculum learning [1] or transfer learning strategy [16] is suitable for the case of low payload steganalysis.”
BackgroundRobust random search with scale-free stochastic resetting
Kuśmierz, Ł., & Toyoizumi, T. (2019) · Physical review. E · doi:10.1103/physreve.100.032110 · Cited by 26
“[61] Y.”
BackgroundEnergy-Based Hindsight Experience PrioritizationPreprint
Zhao, R., & Tresp, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.01363 · Cited by 85
“Our method can be considered as a form of curriculum learning [34, 35, 36, 37, 38, 39].”
MethodImproving Adversarial Robustness Through Progressive HardeningPreprint
Sitawarin, C., Chakraborty, S., & Wagner, D. (2020) · arXiv (Cornell University) · Cited by 31
Progressive modality cooperation for multi-modality domain adaptation
Zhang, W., Xu, D., Zhang, J., & Ouyang, W. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3052083 · Cited by 25
“To reduce domain distribution mismatch, we propose to progressively select highly-reliable target samples in an easy-to-hard fashion similar to curriculum learning [44] and self-paced learning [18].”
BackgroundMethodLearning to learn by using nonequilibrium training protocols for adaptable materials
Falk, M. J., Wu, J., Matthews, A., Sachdeva, V., Pashine, N., Gardel, M. L., Nagel, S. R., & Murugan, A. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2219558120 · Cited by 35
Real-Time CNN Training and Compression for Neural-Enhanced Adaptive Live Streaming
Jeong, S., Kim, B., Cha, S., Seo, K., Chang, H., Lee, J., Kim, Y., & Noh, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3377372 · Cited by 32
“Therefore, we propose a curriculum-based training approach that gradually increases the difficulty of the training target [56], [57].”
BackgroundDe novo motor learning creates structure in neural activity that shapes adaptation
Chang, J. C., Perich, M. G., Miller, L. E., Gallego, J. Á., & Clopath, C. (2024) · Nature Communications · doi:10.1038/s41467-024-48008-7 · Cited by 28
“Task performance is shaped by how behavioral components are acquired 33,34 , including by whether the examples presented during the initial learning process are meaningfully organized with respect to each other 35 .”
Structured output layer with auxiliary targets for context-dependent acoustic modelling
Świętojański, P., Bell, P., & Renals, S. J. (2015) · Interspeech (USB) · doi:10.21437/interspeech.2015-715 · Cited by 25
“All these methods implicitly implement a form of curriculum learning [23] where a lower entropy task (with respect to the complexity of classification task or the number of the optimised weights used for intermediate predictions) is employed to iteratively place some relevant prior on the parameters: for example, by f…”
MethodPornographic image recognition by strongly-supervised deep multiple instance learning
Wang, Y., Jin, X., & Tan, X. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7533195 · Cited by 25
“A number of methods have been proposed for better initialization [11, 12, 13] and better heuristics for local optimization [14, 15, 16].”
MethodEdgeNet: Semantic Scene Completion from a Single RGB- D Image
Dourado, A., de Campos, T. E., Kim, H., & Hilton, A. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413252 · Cited by 33
“For experiments in which we trained our models from scratch, we use the technique known as One Cycle Learning [22], which is a combination of Curriculum Learning [23] and Simulated Annealing [24].”
MethodZhang, C., Li, Y., Chen, X., Jin, Y., Tang, P., & Li, J. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00087 · Cited by 39
“uct an ensemble model with Ksub-models, each of which uses the 30% to 100% of the easiest samples (the samples with lowest losses), which can be regarded as an ensemble version of curriculum learning [28]. Performance metrics Precision: While standard classification problems care about the prediction accuracy on all th…”
InfluentialCILEA-NET: Curriculum-Based Incremental Learning Framework for Remote Sensing Image Classification
Bhat, S. D., Banerjee, B., Chaudhuri, S., & Bhattacharya, A. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3084408 · Cited by 28
“In [11], it was confirmed that training with a curriculum strategy could result in faster training, and lead toward better regions in the parameter space.”
BackgroundElementary Subgraph Features for Link Prediction With Neural Networks
Fang, Z., Tan, S., Wang, Y., & Lü, J. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3132352 · Cited by 26
“The Adam update rule [40] is adopted for optimization with a learning rate of 0.001 and a batch size of 128.”
MethodInfluentialSaigusa, Y., Sakaino, S., & Tsuji, T. (2022) · IEEE Access · doi:10.1109/access.2022.3185651 · Cited by 32
“In this case, curriculum learning [53], [54], which improves the prediction accuracy of the model by initially learning with data under simple conditions and gradually making the model more difficult, could be introduced to”
BackgroundDynamic optimal space partitioning for redirected walking in multi-user environment
Jeon, S.-B., Kwon, S.-U., Hwang, J.-Y., Cho, Y.-H., Kim, H., Park, J., & Lee, I. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530113 · Cited by 28
“Curriculum learning is a technique that trains a model with easy-to-learn data first and then gradually trains it with difficult data [Bengio et al. 2009].”
MethodCurriculum Reinforcement Learning Based on K-Fold Cross Validation
Lin, Z., Lai, J., Chen, X., Cao, L., & Wang, J. (2022) · Entropy · doi:10.3390/e24121787 · Cited by 29
“…approach can be viewed as a special kind of continuous optimization approach that can start with smoother (i.e., simpler) optimization problems and gradually add coarser (i.e., more difficult), non-convex (norm) optimization problems to eventually perform the optimization of the target task [14].”
BackgroundMethodInfluentialSelf-Supervised Keypoint Discovery in Behavioral Videos
Sun, J. J., Ryou, S., Goldshmid, R. H., Weissbourd, B., Dabiri, J. O., Anderson, D. J., Kennedy, A., Yue, Y., & Perona, P. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00221 · Cited by 50
“We adopt curriculum learning [2] and apply Lr and Ls once the keypoints are consistently discovered from the semantic parts of the target instance.”
Identity-Aware Contrastive Knowledge Distillation for Facial Attribute Recognition
Chen, S., Zhu, X., Yan, Y., Zhu, S., Li, S., & Wang, D. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3253799 · Cited by 27
“Motivated by curriculum learning [40], we aim to give initial priority to transferring shallower distillation points, and then gradually focus on the transfer of deeper distillation points.”
BackgroundCode Execution with Pre-trained Language Models
Liu, C., Lu, S., Chen, W., Jiang, D., Svyatkovskiy, A., Fu, S., Sundaresan, N., & Duan, N. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.308 · Cited by 51
Meta Compositional Referring Expression Segmentation
Xu, L., Huang, H., Shang, X., Yuan, Z., Sun, Y., & Liu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01866 · Cited by 47
“Besides, given that the various levels of novel compositions correspond to different levels of comprehension complexity, we adopt a curriculum learning strategy [2, 33] for model training, so that the model can progressively learn to handle various levels of novel compositions, i.”
Vehicle dynamic dispatching using curriculum-driven reinforcement learning
Zhang, X., Xiong, G., Ai, Y., Liu, K., & Chen, L. F. (2023) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2023.110698 · Cited by 36
Power SVM: Generalization with exemplar classification uncertainty
Zhang, W., Yu, S. X., & Teng, S. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247921 · Cited by 28
Self-paced and soft-weighted nonnegative matrix factorization for data representation
Huang, S., Zhao, P., Ren, Y., et al. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.10.003 · Cited by 25
Beyond Weakly Supervised: Pseudo Ground Truths Mining for Missing Bounding-Boxes Object Detection
Zhang, Y., Ding, M., Bai, Y., Xu, M., & Ghanem, B. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2898559 · Cited by 29
Assisted Excitation of Activations: A Learning Technique to Improve Object Detectors
Derakhshani, M. M., Masoudnia, S., Shaker, A. H., Mersa, O., Sadeghi, M. A., Rastegari, M., & Araabi, B. N. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00942 · Cited by 36
“This strategy is inspired by curriculum learning [2]; it simplifies the task of detection and localization in the early stages of training and gradually makes the task more difficult and realistic.”
Kurniawan, B., Vamplew, P. W., Papasimeon, M., Dazeley, R., & Foale, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-35288-2_5 · Cited by 30
Handwritten Historical Music Recognition by Sequence-to-Sequence with Attention Mechanism
Baró, A., Badal, C., & Fornés, A. (2020) · International Conference on Frontiers in Handwriting Recognition · doi:10.1109/icfhr2020.2020.00046 · Cited by 25
“We train our model using curriculum learning [28] for improving the performance.”
MethodTraining Recurrent Neural Networks by DiffusionPreprint
Mobahi, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.04114 · Cited by 46
“…error) gradient based optimization methods are often used in combination with other techniques such as smart initialization [Sutskever et al., 2013], layer-wise pretraining [Bengio et al., 2007], dropout [Hinton et al., 2012b], annealed learning rate, and curriculum learning [Bengio et al., 2009].”
MethodShirahama, K., & Grzegorzek, M. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-014-2292-8 · Cited by 24
“In [18], a two-step curriculum was used for classification of shapes (rectangles, ellipses and triangles).”
BackgroundMethodInfluentialCurriculum Based Reinforcement Learning of Grid Topology Controllers to Prevent Thermal Cascading
Matavalam, A. R. R., Guddanti, K. P., Weng, Y., & Ajjarapu, V. (2022) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2022.3213487 · Cited by 37
“Recent approaches [20] have proposed to learn curriculum strategies as a part of the overall ML-based approach for classification or regression tasks.”
BackgroundUnderstanding the development of reward learning through the lens of meta-learning
Nussenbaum, K., & Hartley, C. A. (2024) · Nature Reviews Psychology · doi:10.1038/s44159-024-00304-1 · Cited by 45
CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learning
Chane-Sane, E., Léziart, P., Flayols, T., Stasse, O., Souères, P., & Mansard, N. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802334 · Cited by 62
“In this work, we follow recent successful approaches based on model-free RL [8] to train policies on a curriculum of increasingly difficult settings [9], [10] in simulation and directly transfer the learned policy on the physical robot [11], [12], [13] to overcome challenging obstacles.”
Du, K.-L., Zhang, R., Jiang, B., Zeng, J., & Lu, J. (2025) · Mathematics · doi:10.3390/math13030451 · Cited by 44
Kozlowski, A. C., & Evans, J. A. (2025) · Sociological Methods & Research · doi:10.1177/00491241251337316 · Cited by 72
Adaptive Dispersal and Collaborative Clustering for Few-Shot Unsupervised Domain Adaptation
Lu, Y., Huang, H., Wong, W. K., Hu, X. L., Lai, Z., & Li, X. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3581007 · Cited by 38
“Hence, by adjusting the mix-up ratio adaptively in a curriculum learning way [47], the domain alignment in FUDA goes more smoothly, and the model is more robust in domain-irrelevant information.”
Object and Action Classification with Latent Window Parameters
Bilen, H., Namboodiri, V. P., & Gool, L. V. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0646-8 · Cited by 29
“Recent work (Bengio et al. 2009) proposes an iterative approach to find better local minima within shorter convergence times for non-convex optimization problems.”
BackgroundWu, Y., Shah, S. K., & Kakadiaris, I. A. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.12.002 · Cited by 24
“During training, inspired by curriculum learning [37], we gradually increase the value of β and encourage the network to discriminate pixels closer to the key-point locations.”
BackgroundLarge Scale Landmark Recognition via Deep Metric Learning
Boiarov, A., & Tyantov, E. (2019) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3357384.3357956 · Cited by 33
“Significantly more successful was the strategy inspired by a curriculum learning [1], that made it”
MethodImproved curriculum learning using SSM for facial expression recognition
Liu, X., & Zhou, F. (2019) · The Visual Computer · doi:10.1007/s00371-019-01759-7 · Cited by 27
“[21] first proposed the concept of curriculum learning.”
BackgroundA Curiosity-Based Learning Method for Spiking Neural Networks
Shi, M., Zhang, T., & Zeng, Y. (2020) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2020.00007 · Cited by 23
“Bengio et al. (2009) proposed curriculum learning to imitate the characteristics of human learning process, and let the model learn from simple to difficult in multiple stages (Ugur et al., 2007; Chernova and Veloso, 2009).”
BackgroundLearning Generalized Relational Heuristic Networks for Model-Agnostic Planning
Karia, R., & Srivastava, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.16983 · Cited by 50
“Curriculum learning (Bengio et al. 2009) shows that effective learning is possible by organizing the training data in the form of a schedule.”
BackgroundJoint Space Control via Deep Reinforcement Learning
Kumar, V. C. V., Hoeller, D., Sundaralingam, B., Tremblay, J., & Birchfield, S. T. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636477 · Cited by 36
A Competence-Aware Curriculum for Visual Concepts Learning via Question Answering
Li, Q., Huang, S., Hong, Y., & Zhu, S. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_9 · Cited by 33
“The competence-aware curriculum in our work is related to curriculum learning [7,54,56,20,51,45,21,48] and machine teaching [61,62,39,11,41,15,58].”
BackgroundContinuation Multiple Instance Learning for Weakly and Fully Supervised Object Detection
Ye, Q., Wan, F., Liu, C., Huang, Q., & Ji, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3070801 · Cited by 28
“Curriculum learning [35] shared a similar principle by defining”
BackgroundEmbedding Probability Guided Network for Image Steganalysis
Li, Q., Feng, G., Ren, Y., et al. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3083546 · Cited by 24
“For payload 0.2 bpnzac, a similar curriculum learning strategy [22] is adopted.”
MethodInfluentialRule mining over knowledge graphs via reinforcement learning
Chen, L., Jiang, S., Liu, J., Wang, C., Zhang, S., Xie, C., Liang, J., Xiao, Y., & Song, R. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108371 · Cited by 34
Dynamic Curriculum Learning for Great Ape Detection in the Wild
Yang, X., Burghardt, T., & Mirmehdi, M. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01748-3 · Cited by 29
“The CL training approach (Bengio et al. 2009) has had significant impact on the design of computer vision algorithms, such as (Karras et al. 2018; Wang et al. 2018; Huang et al. 2020; Wang et al. 2022; Zhang et al. 2021).”
BackgroundWeakly Supervised Object Detection for Remote Sensing Images: A Survey
Fasana, C., Pasini, S., Milani, F., & Fraternali, P. (2022) · Remote Sensing · doi:10.3390/rs14215362 · Cited by 29
Curriculum-Based Asymmetric Multi-Task Reinforcement Learning
Huang, H., Ye, D., Shen, L., & Liu, W. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3223872 · Cited by 33
“The idea of using curricula to train learning agents was first proposed in [11] and first introduced for deep learning by [5].”
BackgroundActive learning with deep autoencoders for seismic facies interpretation
Mustafa, A., & AlRegib, G. (2023) · Geophysics · doi:10.1190/geo2022-0353.1 · Cited by 24
Curriculum label distribution learning for imbalanced medical image segmentation
Li, X., Luo, G., Wang, W., Wang, K., & Li, S. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102911 · Cited by 26
Shi, J., Li, K., Piao, C., Gao, J., & Chen, L. (2023) · Sensors · doi:10.3390/s23167124 · Cited by 29
Single image super-resolution approaches in medical images based-deep learning: a survey
El‐Shafai, W., Ali, A. M., El‐Nabi, S. A., El‐Rabaie, E. M., & El‐Samie, F. E. A. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16197-w · Cited by 29
“The curriculum learning approach [15] responds to the varying difficulty of tasks, starting with processing a little noise in basic images and then progressing to processing detailed images.”
Toward Effective Semi-supervised Node Classification with Hybrid Curriculum Pseudo-labeling
Luo, X., Ju, W., Gu, Y., Qin, Y., Yi, S., Wu, D., Liu, L., & Zhang, M. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3626528 · Cited by 34
“Note that curriculum learning [4] is a training strategy that trains a learning model from easier data to harder data.”
BackgroundMosqueira-Rey, E., Hernández-Pereira, E., Bobes-Bascarán, J., Alonso-Ríos, D., Pérez-Sánchez, A., Fernández-Leal, Á., Moret‐Bonillo, V., Vidal-Ínsua, Y., & Vázquez-Rivera, F. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09197-2 · Cited by 26
“Here, we find curriculum learning (CL) [10], in which the dataset is organized in terms of increasing complexity in order to take advantage of previously learned concepts and to ease the abstraction of new concepts.”
Liu, Y., Wang, J., Xiao, L., Liu, C., Wu, Z., & Xu, Y. D. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3331778 · Cited by 33
“Curriculum learning is a machine learning paradigm proposed by Bengio et al. [32].”
Multi-modal self-paced learning for image classification
Xu, W., Liu, W., Huang, X., Yang, J., & Qiu, S. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.04.075 · Cited by 26
Lalor, J. P., Wu, H., Munkhdalai, T., & Yu, H. Q. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1500 · Cited by 49
“Our result is also consistent with the key assumptions of curriculum learning (Bengio et al., 2009).”
ResultChange Detection from Remote Sensing to Guide OpenStreetMap Labeling
Albrecht, C. M., Zhang, R., Cui, X., Freitag, M., Hamann, H. F., Klein, L. J., Finkler, U., Marianno, F. J., Schmude, J., Bobroff, N., et al. (2020) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi9070427 · Cited by 24
“Moreover, proper training of ANNs led to advanced optimization techniques which were discussed in [96,97].”
MethodPILOT: Efficient Planning by Imitation Learning and Optimisation for Safe Autonomous Driving
Pulver, H., Eiras, F., Carozza, L., Hawasly, M., Albrecht, S. V., & Ramamoorthy, S. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636862 · Cited by 32
“After many failed attempts at training with all constraints from a random initialisation, we applied a curriculum learning approach [37], sequentially introducing constraints and”
Method
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.