Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 2,701–2,800 of 5,240
Curriculum Dataset Distillation
Ma, Z., Cao, A., Yang, F., Gong, Y., & Wei, X. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3579228 · Cited by 22
“This strategy, grounded in the easy-to-hard training paradigm common in curriculum learning [33], [34], [35], [36], presents multiple advantages: it ensures synthetic image diversity by progressively increasing di ffi culty, facilitates foundational feature learning with simpler images initially…”
Lin, Y., Zhang, R., Huang, W., Wang, K., Ding, Z., So, D. K. C., & Niyato, D. T. (2025) · IEEE Transactions on Communications · doi:10.1109/tcomm.2025.3597815 · Cited by 15
“In addition to the dataset, we introduce a fine-tuning approach guided by Pointwise V-Information (PVI) metrics inspired by curriculum learning [11], [12].”
Learning Robust Adaptive Bitrate Algorithms with Adversarial Inverse Reinforcement Learning
Yi, L., & Qin, Y. (2025) · Chinese Journal of Electronics · doi:10.23919/cje.2024.00.202 · Cited by 6
Web-scale Multimedia Search for Internet Video Content
Jiang, L. (2016) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2835776.2855081 · Cited by 5
“The theory is inspired by the cognitive processes of humans and animals, which generally start with learning easier aspects of a task, and then gradually consider more complex examples [2, 15].”
BackgroundAziz, Z., Mohamad, S. H., & Rahmat, R. (2011) · World Applied Sciences Journal · Cited by 5
3D Saliency for Finding Landmark Buildings
Kobyshev, N., Riemenschneider, H., Bódis-Szomorú, A., & Gool, L. V. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.35 · Cited by 9
“Related is also [28], who use curriculum learning [3] for unsupervised object discovery by starting with the easiest clusters and gradually increasing the complexity of clusters.”
BackgroundNo free lunch in ball catching: A comparison of Cartesian and angular representations for control
Höfer, S., Raisch, J., Toussaint, M., & Brock, O. (2018) · PLoS ONE · doi:10.1371/journal.pone.0197803 · Cited by 8
Approximation of Time-Consuming Simulation Based on Generative Adversarial Network
Orihara, R., Narasaki, R., Yoshinaga, Y., et al. (2018) · IEEE Annual Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac.2018.10223 · Cited by 5
“The parental sampling can be seen as a form of curriculum learning [10] because the additional information given by the supervised term is available in the early stage of the learning.”
BackgroundStudying the impact of the Full-Network embedding on multimodal pipelines
Vilalta, A., García-Gasulla, D., Parés, F., Ayguadé, E., Labarta, J., Moya-Sánchez, E. U., & Cortés, U. (2018) · Semantic Web · doi:10.3233/sw-180341 · Cited by 7
“They indicate that a rough form of curriculum learning [2] could be applied, but do not develop or experiment it further as in their preliminary experiments it obtained worse performance than the proposed method.”
ResultDCSR: Deep clustering under similarity and reconstruction constraints
Yu, L., & Wang, W. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.06.013 · Cited by 5
Reinforcement Learning Aided Network Architecture Generation for JPEG Image Steganalysis
Yang, J., Lu, B., Xiao, L., et al. (2020) · ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) · doi:10.1145/3369412.3395060 · Cited by 5
“All networks use curriculum training [5] to reduce the experiment time, and use the same fine-tuning order as in SRNet.”
MethodAutoPath: Image-Specific Inference for 3D Segmentation
Sun, D., Wang, Y., Ni, D., & Wang, T. (2020) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2020.00049 · Cited by 5
“To achieve efficient training, we further employ curriculum learning (Bengio et al., 2009) to train the policy network.”
MethodSurrogate dropout: Learning optimal drop rate through proxy
Hu, J., Chen, Y., Zhang, L., & Zhang, Y. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106340 · Cited by 6
Label Noise Robust Curriculum for Deep Paraphrase Identification
Li, B., Liu, T., Wang, B., & Wang, L. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206712 · Cited by 5
“Few methods have been proposed to leverage curriculum learning [18] to train robust deep models, which are inspired by studies from Arpit et al.”
MethodLearning When to Switch: Composing Controllers to Traverse a Sequence of Terrain Artifacts
Tidd, B., Cosgun, A., Leitner, J., & Hudson, N. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636233 · Cited by 7
“Guided Curriculum Learning: Curriculum learning is a systematic way of increasing the difficulty of a task [31], and results in learning attractive walking gaits quickly [32].”
Chen, Z., Wang, Y., Lin, D., Cheng, D. Z., Hong, L., Ed, H., & Cui, C. M. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441770 · Cited by 12
“In curriculum learning [1], prediction variation can be used as a way for estimating example difficulty.”
MethodSuro, F., Ferber, J., Stratulat, T., & Michel, F. (2021) · Autonomous Robots · doi:10.1007/s10514-020-09960-7 · Cited by 8
“Curriculum learning (Bengio et al. 2009) is a progressive learning method investigated to this day in the field of”
BackgroundMethodInter-domain curriculum learning for domain generalization
Kim, D., Kim, J., & Lee, J. (2021) · ICT Express · doi:10.1016/j.icte.2021.11.009 · Cited by 4
Data Augmentation for Text Generation Without Any Augmented Data
Bi, W., Li, H., & Huang, J. (2021) · doi:10.18653/v1/2021.acl-long.173 · Cited by 9
“The sample weights are often carefully defined (Freund and Schapire, 1997; Bengio et al., 2009) or learnt by another network (Jiang et al.”
BackgroundExploiting abstractions for grammar‐based learning of complex multi‐agent behaviours
Samarasinghe, D., Barlow, M., Lakshika, E., & Kasmarik, K. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22550 · Cited by 6
“Curriculum learning 17 is a relatively recent idea, which is an extension of incremental learning and is inspired by the curriculum and organisation of the education system which progressively introduces different concepts and examples.”
BackgroundRobotic Occlusion Reasoning for Efficient Object Existence Prediction
Li, M., Weber, C., Kerzel, M., Lee, J. H., Zeng, Z., Liu, Z., & Wermter, S. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635947 · Cited by 5
“The model can be jointly trained with reinforcement learning and supervised learning methods using a curriculum training strategy [6].”
MethodSelf-Supervised Deep Fisheye Image Rectification Approach using Coordinate Relations
Hosono, M., Simo‐Serra, E., & Sonoda, T. (2021) · doi:10.23919/mva51890.2021.9511349 · Cited by 5
Generalization Self-distillation with Epoch-wise Regularization
Xia, Y., & Yang, Y. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533993 · Cited by 5
“In our work, cumulative learning strategy [26, 27] is used to shift the learning focus between truth one-hot vector y and teacher predictions p t − 1 .”
MethodFingerspelling Recognition in the Wild with Fixed-Query based Visual Attention
S, S. K. S., Jose, G., Tandon, N., et al. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475580 · Cited by 5
“Inspired from the curriculum learning approach [3], the training is performed in two stages - first with simpler video sequences and finally using the entire dataset to ensure better convergence.”
MethodRecognizing Social Signals with Weakly Supervised Multitask Learning for Multimodal Dialogue Systems
Hirano, Y., Okada, S., & Komatani, K. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479927 · Cited by 8
“The strategy that learns simpler samples before more difficult samples is called curriculum learning [5].”
BackgroundIwamoto, S., Raytchev, B., Tamaki, T., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87735-4_4 · Cited by 6
“In stage 2 of the proposed method, we utilize Curriculum Learning [2] to leverage the information in the Uncertainty Map about the uncertainty of the model to improve its reliability (methods for evaluating the reliability of a model are explained in Section 3).”
BackgroundMethodWeakly-Supervised Free Space Estimation through Stochastic Co-Teaching
Robinet, F., Parera, C., Hundt, C., & Frank, R. (2022) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw54805.2022.00068 · Cited by 5
“Curriculum learning [4] is based on training a model on samples of increasing difficulty, which can correspond to different noise levels [24, 15].”
BackgroundA Deep Safe Reinforcement Learning Approach for Mapless Navigation
Lv, S., Li, Y., Liu, Q., Gao, J., Pang, X., & Chen, M. (2021) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio54168.2021.9739251 · Cited by 5
“We adopt the curriculum learning [30] approach to train our model, which can ease the difficulty level of the training to speed up the training process and make the model generalize better.”
MethodTD-Sampler: Learning a Training Difficulty Based Sampling Strategy for Few-Shot Object Detection
Wu, C., Wang, B., Liu, S., et al. (2022) · International Conference on Cloud Computing and Big Data Analytics (ICCCBDA) · doi:10.1109/icccbda55098.2022.9778859 · Cited by 5
Bortoloti, F. D., Tavares, J., Rauber, T. W., Ciarelli, P. M., & Botelho, R. C. G. (2022) · Machine Vision and Applications · doi:10.1007/s00138-022-01335-5 · Cited by 7
“The process used for learning, called curriculum learning [2], consists in feeding the network with weakly annotated images in order from easy to hard.”
MethodShin, C., Ryu, H., Cho, E., Han, S., Lee, K., Kim, C., & Yoon, Y. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102600 · Cited by 5
ConCS: A Continual Classifier System for Continual Learning of Multiple Boolean Problems
Nguyen, T. B., Browne, W. N., & Zhang, M. (2022) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2022.3210872 · Cited by 5
“II) from agent indexed from [0] to [18].”
BackgroundCES-KD: Curriculum-based Expert Selection for Guided Knowledge Distillation
Amara, I., Ziaeefard, M., Meyer, B. H., Gross, W. J., & Clark, J. J. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956061 · Cited by 9
“It was shown that this technique efficiently enhances the performance of deep models [16].”
MethodCurriculum Adversarial Training for Robust Reinforcement Learning
Sheng, J., Zhai, P., Dong, Z., Kang, X., Chen, C., & Zhang, L. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892908 · Cited by 12
“In 2009, Bengio et al. [14] proposed the original concept of curriculum learning.”
BackgroundMethodAlcover‐Couso, R., SanMiguel, J. C., Escudero-Viñolo, M., & García‐Martín, Á. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14662-0 · Cited by 8
“In different computer vision fields such as object localization, some works obtain state of the art performance by defining an easy to hard presentation commonly known as curriculum learning [2].”
BackgroundInfluentialCurriculum Based Multi-Task Learning for Parkinson’s Disease Detection
Dhinagar, N. J., Owens‐Walton, C., Laltoo, E., Boyle, C. P., Chen, Y.-L., Cook, P. A., McMillan, C. T., Tsai, C., Wang, J.-J., Wu, Y., et al. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230355 · Cited by 7
“presented one of the earlier works on the benefits of curriculum learning on the training process [10].”
BackgroundStone masonry design automation via reinforcement learning
Kang, S., Dy, J., & Kane, M. B. (2023) · Artificial intelligence for engineering design analysis and manufacturing · doi:10.1017/s0890060423000100 · Cited by 12
“Another direction lies in the use of pre-trained models through transfer learning (Torrey and Shavlik, 2010) or curriculum learning (Bengio et al., 2009).”
A Curriculum Framework for Autonomous Network Defense using Multi-agent Reinforcement Learning
Campbell, R. G., Eirinaki, M., & Park, Y. (2023) · Silicon Valley Cybersecurity Conference (SVCC) · doi:10.1109/svcc56964.2023.10165310 · Cited by 7
“…leads to forming part of a training curriculum, as we can increase the standard deviation of the parameters to train the agents with progressively more challenging environments [20] ranging from low severity (low exploitability-low impact) up to critical severity (high exploitability-high impact).”
BackgroundDeep ensemble-based hard sample mining for food recognition
Nagarajan, B., Bolaños, M., Aguilar, E., & Radeva, P. I. (2023) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2023.103905 · Cited by 5
Pavlova, V., & Makhlouf, M. (2023) · Workshop on Biomedical Natural Language Processing · doi:10.18653/v1/2023.bionlp-1.31 · Cited by 3
“2 Prior Literature El-man, 1993; Bengio et al., 2009) has been successfully explored in NLP to solve different tasks.”
BackgroundSearch Query Spell Correction with Weak Supervision in E-commerce
Kakkar, V., Sharma, C., Pande, M., & Kumar, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-industry.66 · Cited by 10
“Curriculum learning (Bengio et al., 2009; Elman, 1993) has shown to generate robust models which show the improvements in their learning ability when they first learn on simpler tasks followed by tougher tasks.”
BackgroundLow-Bandwidth Self-Improving Transmission of Rare Training Data
George, S., Turki, H., Feng, Z., Ramanan, D., Pillai, P. S., & Satyanarayanan, M. (2023) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3570361.3613300 · Cited by 10
Watch Where You Head: A View-biased Domain Gap in Gait Recognition and Unsupervised Adaptation
Habib, G., Barzilay, N., Shimshi, O., Ben‐Ari, R., & Darshan, N. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00600 · Cited by 8
“Inspired by [2, 47], we apply the Triplet Selection al-gorithm in four different stages during training, in a from-1 In the supplementary materials, we present further intuition and justification, including a shared visualization of views and identities.”
MethodAutomatic Curriculum Determination for Deep Reinforcement Learning in Reconfigurable Robots
Karni, Z., Simhon, O., Zarrouk, D., & Berman, S. (2024) · IEEE Access · doi:10.1109/access.2024.3406768 · Cited by 5
“In curricula arranged by robot operations, the robot’s mission is divided into distinct operations [12] via manual segmentation, interactive feedback, or prior knowledge about the mission and the operations.”
BackgroundMethodAn Invitation to Deep Reinforcement Learning
Jaeger, B., & Geiger, A. (2024) · Foundations and Trends® in Optimization · doi:10.1561/2400000049 · Cited by 18
“Another idea to make sparse reward settings or more generally hard tasks easier to learn is curriculum learning (Selfridge et al., 1985; Krueger & Dayan, 2009; Bengio et al., 2009).”
BackgroundImproving Source Separation via Multi-Speaker Representations
Zegers, J., & hamme, H. V. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-754 · Cited by 6
“The networks were trained using curriculum learning [26], i.”
MethodTree-Structured Curriculum Learning Based on Semantic Similarity of Text
Han, S., & Myaeng, S.-H. (2017) · IEEE International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla.2017.00-27 · Cited by 5
“In developing an effective sequential CL method that follows Definition 1, the variability measure plays an important role [1].”
BackgroundMethodInfluentialSalient object detection via proposal selection
Zhang, L., & Zhou, Q. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.01.050 · Cited by 5
Continuous Trade-off Optimization Between Fast and Accurate Deep Face Detectors
Soviany, P., & Ionescu, R. T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-04179-3_42 · Cited by 5
“Bengio et al. [1] have explored easy-to-hard strategies to train machine learning models, showing that machines can also benefit from learning by gradually adding more difficult examples.”
BackgroundTraining-ValueNet: Data Driven Label Noise Cleaning on Weakly-Supervised Web Images
Smyth, L., Kangin, D., & Pugeault, N. (2019) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2019.8850689 · Cited by 4
“[18]”
Multi-focus image fusion via NSST with non-fixed base dictionary learning
Yang, Y., Zhang, Y., Huang, S., et al. (2019) · International Journal of Systems Assurance Engineering and Management · doi:10.1007/s13198-019-00887-6 · Cited by 5
“However, the Bengio et al. (2009) confirmed that SF can better extract the clear part of the source image than the other two criteria.”
BackgroundChang, H., Xu, Z., & Tomizuka, M. (2020) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2020.12.2317 · Cited by 7
“There are other methods seeking to learn a globally general policy: Curriculum learning (CL)(Bengio et al. (2009)) trains a model on a sequence of cognate tasks that get more and more challenging gradually, so as to solve hard tasks that could not be learned from scratch.”
BackgroundArtifact-Free Single Image Defogging
Graffieti, G., & Maltoni, D. (2021) · Atmosphere · doi:10.3390/atmos12050577 · Cited by 5
“In order to train a model with real images and at the same time reduce the artifacts inserted during the defogging, we propose CurL-Defog, an approach inspired by curriculum learning [13], where the model is first guided towards a desirable parameter-space region via pix2pix-like [34] training using a synthetic paired…”
MethodTransfer Learning for Sequence Generation: from Single-source to Multi-source
Huang, X., Xu, J., Sun, M., & Liu, Y. (2021) · doi:10.18653/v1/2021.acl-long.446 · Cited by 6
“Curriculum learning (Bengio et al., 2009) aims to learn from examples organized in an easy-tohard order, and intermediate tasks (Pruksachatkun et al.”
BackgroundImproving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning
Zhang, X., Yu, B., Liu, T., Zhang, Z., Sheng, J., Xue, M., & Xu, H. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.131 · Cited by 5
Carvalho, R., Morgado, A. C., Andrade, C., Nedelcu, T., Carreiro, A. V., & Vasconcelos, M. J. M. (2021) · Diagnostics · doi:10.3390/diagnostics12010036 · Cited by 5
“This learning procedure is called curriculum learning [32] and has been recently considered in the medical image field.”
BackgroundGraph Neural Network based Child Activity Recognition
Mohottala, S., Samarasinghe, P. D., Kasthurirathna, D., & Abhayaratne, C. (2022) · IEEE International Conference on Industrial Technology (ICIT) · doi:10.1109/icit48603.2022.10002799 · Cited by 6
“3) Curriculum Learning: To improve the efficiency of both Standard Deep Learning and TL, Curriculum Learning (CL) [46] was applied, where the model was trained starting from easy samples and gradually exposing to more challenging”
MethodJoint learning of ontology and semantic parser from text
Starc, J., & Mladenić, D. (2017) · Intelligent Data Analysis · doi:10.3233/ida-150452 · Cited by 5
“The grammar induction method works on the premise of curriculum learning [2], where the parser first learns to parse simple sentences, then proceeds to learn more complex ones.”
Web-scale Multimedia Search for Internet Video Content
Jiang, L. (2016) · doi:10.1145/2872518.2888599 · Cited by 5
“The theory is inspired by the cognitive processes of humans and animals, which generally start with learning easier aspects of a task, and then gradually consider more complex examples [2, 15].”
BackgroundComposite Denoising Autoencoders
Geras, K. J., & Sutton, C. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46128-1_43 · Cited by 7
“Firstly, our training procedure can be considered an application of the idea of curriculum learning [4, 14].”
MethodHidden Layers in Perceptual Learning
Cohen, G., & Weinshall, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.568 · Cited by 7
“This is reminiscent of similar phenomena in cognitive learning, and the related concept of curriculum learning [4].”
BackgroundResultRobust nonlinear adaptation algorithms for multitask prediction networks
Abuduweili, A., & Liu, C. (2020) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.3198 · Cited by 6
Unsupervised learning of probabilistic grammars
Tu, K. (2012) · doi:10.31274/etd-180810-585 · Cited by 5
Improving Human Activity Recognition Through Ranking and Re-rankingPreprint
Lan, Z., Yu, S.-I., & Hauptmann, A. G. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.03740 · Cited by 5
“The second method we suggest is inspired by curriculum learning which mimics the human learning scheme and has become popular for image and video classification [3, 5, 11, 6].”
BackgroundMethodInfluentialExploiting Web Images for Weakly Supervised Object DetectionPreprint
Tao, Q., Yang, H., & Cai, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.08721 · Cited by 5
“Curriculum Learning: Our work is inspired by curriculum learning [1] scheme.”
MethodHWNet v2: An Efficient Word Image Representation for Handwritten DocumentsPreprint
Krishnan, P., & Jawahar, C. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06194 · Cited by 5
“To avoid such scenarios, we use the strategy from curriculum learning [8], where we start the training process from synthetic images showing easy examples (based on the number of characters) first and harder later.”
MethodStarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer LearningPreprint
Shao, K., Zhu, Y., & Zhao, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00810 · Cited by 5
“The initial tasks are used to guide the learner so that it can perform better on the final task [53].”
MethodMultimodal Co-Training for Selecting Good Examples from Webly Labeled VideoPreprint
Hinami, R., Liang, J., Satoh, S., & Hauptmann, A. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.06057 · Cited by 12
“λare used for training. v is initialized by the pseudo label’s confidencev0 computed by word matching between the concept name and metadata, which corresponds to the intuition of curriculum learning [2] that determines the learning sequence from external prior knowledge. WELL determines the learning sequence by using…”
BackgroundMethodAutomatic understanding of unwritten languages
Adams, O. (2017) · Minerva Access (University of Melbourne) · Cited by 92
Bai, H., Zhou, Y., Zhang, J., Zhao, L., Hwang, M.-Y., & Zong, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.06167 · Cited by 9
“As our SCRT only optimizes the SKR score with monolingual data, the curriculum learning (Bengio et al., 2009) strategy is employed in which we begin with the class-based model, and gradually increase the number of SCRT training steps.”
MethodSelf-Paced Learning with Adaptive Deep Visual EmbeddingsPreprint
Thangarasa, V., & Taylor, G. W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.09200 · Cited by 6
“Previous studies on curriculum learning (CL) [3] show that organizing training samples based on the ascending order of difficulty can be favourable for model training.”
BackgroundElastic Boundary Projection for 3D Medical Image SegmentationPreprint
Ni, T., Xie, L., Zheng, H., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00518 · Cited by 5
“To alleviate this issue, in the training stage, we gradually replace the added term in Eqn (4) with prediction, following the idea of curriculum learning [1].”
MethodTheory of Curriculum Learning, with Convex Loss FunctionsPreprint
Weinshall, D., & Amir, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03472 · Cited by 27
“Early in the training procedure we expect Case 1 , when cos ϑ > 0 and ¯ w , w t are positively correlated, to dominate SGD learning.”
BackgroundMethodInfluentialGârbacea, C., Carton, S., Yan, S., & Mei, Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00398 · Cited by 4
Soft Label Memorization-Generalization for Natural Language InferencePreprint
Lalor, J. P., Wu, H., & Yu, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08563 · Cited by 17
Unsupervised Tracklet Person Re-IdentificationPreprint
Li, M., Zhu, X., & Gong, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.00535 · Cited by 5
“Hence, the CCTA loss is in a curriculum leaning spirit [58].”
BackgroundAutoAssist: A Framework to Accelerate Training of Deep Neural NetworksPreprint
Zhang, J., Yu, H., & Dhillon, I. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.03381 · Cited by 24
“For example, curriculum learning (CL) [3], which presents easier instances to the model before hard ones, was shown to be beneficial to the overall convergence; however, prior knowledge of the training set is required to decide the curriculum.”
BackgroundMulti-Task Curriculum Transfer Deep Learning of Clothing AttributesPreprint
Dong, Q., Gong, S., & Zhu, X. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.03670 · Cited by 5
Video Jigsaw: Unsupervised Learning of Spatiotemporal Context for Video Action RecognitionPreprint
Ahsan, U., Madhok, R., & Essa, I. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07507 · Cited by 5
“Progressive Training Approach We borrow principles from curriculum learning [6] to train our video jigsaw network with an easy jigsaw puzzle task first and then train it for a harder task.”
MethodCurriculum Learning of Multiple TasksPreprint
Pentina, A., Sharmanska, V., & Lampert, C. H. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1353 · Cited by 5
“In [4] Bengio et al. showed experimentally that choosing training examples in an order of gradually increasing difficulty can lead to faster training and higher prediction quality.”
BackgroundCurriculum Learning for Domain Adaptation in Neural Machine TranslationPreprint
Zhang, X., Shapiro, P., Kumar, G., McNamee, P., Carpuat, M. J., & Duh, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05816 · Cited by 12
“In contrast, the probabilistic curriculum (Bengio et al., 2009) works by dividing the training procedure into distinct phases.”
BackgroundMethodDeepCrawl: Deep Reinforcement Learning for Turn-based Strategy GamesPreprint
Sestini, A., Kuhnle, A., & Bagdanov, A. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.01914 · Cited by 10
“In preliminary experiments we noticed that agents learned very slowly, and so to aid the training and overcome the problem of the sparse reward function, we use curriculum learning (Bengio et al. 2009) with phases shown in figure 3.”
MethodGeneralization in Transfer LearningPreprint
Ada, S. E., Uğur, E., & Akın, H. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01331 · Cited by 5
“at focuses on discovering the optimal arrangement of the source tasks to perform better on the target task. To excel in complex tasks, humans follow specifically designed curricula in higher education [35]. Similar to how a more personalized curriculum leads to a more successful result for humans, this strategy is bene…”
BackgroundConvolutional Neural Networks with Dynamic RegularizationPreprint
Wang, Y., Bian, Z.-P., Hou, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11862 · Cited by 5
“, the curriculum learning [14], [21] or selfpaced learning [26]).”
BackgroundStochastic Optimal Control as Approximate Input InferencePreprint
Watson, J., Abdulsamad, H., & Peters, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.03003 · Cited by 33
“This effect bares a resemblance to curriculum learning [24], where the task (e.g. cost function) increases in difficulty as the performance improves, as a strategy for learning complex tasks effectively.”
BackgroundLanguage Grounding through Social Interactions and Curiosity-Driven Multi-Goal LearningPreprint
Lair, N., Colas, C., Portelas, R., Dussoux, J.-M., Dominey, P. F., & Oudeyer, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03219 · Cited by 9
“Curriculum learning refers to the organization of a learning trajectory into distinct training phases, usually starting simple and gradually complexifying the task [16; 17; 18].”
BackgroundContinual egocentric object recognitionPreprint
Erculiani, L., Giunchiglia, F., & Passerini, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.05029 · Cited by 5
“• Progressively introduce novel objects in a developmental fashion [3, 31], and provide supervision on-demand in an online active learning fashion.”
BackgroundNeural Data Server: A Large-Scale Search Engine for Transfer Learning DataPreprint
Yan, X., Acuna, D., & Fidler, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.02799 · Cited by 5
“In active learning [42] one searches over unlabeled data to find optimal samples to be labeled by an oracle, while in curriculum learning [6] subsets of data of increasing difficulty are sought for during training.”
BackgroundDifferentiable Bandit ExplorationPreprint
Boutilier, C. E., Hsu, C., Kveton, B., Mladenov, M., Szepesvári, C., & Zaheer, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06772 · Cited by 8
“Thus, we propose an additional variance reduction technique, which is motivated by curriculum learning (Bengio et al., 2009).”
MethodA Visual Communication Map for Multi-Agent Deep Reinforcement LearningPreprint
Nguyen, N. D., Nguyen, T. T., Creighton, D., & Nahavandi, S. (2020) · arXiv (Cornell University) · doi:10.13140/rg.2.2.13433.62563 · Cited by 9
“This approach is called curriculum learning [40], [41].”
BackgroundGenerating Automatic Curricula via Self-Supervised Active Domain RandomizationPreprint
Raparthy, S. C., Mehta, B., Golemo, F., & Paull, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.07911 · Cited by 12
“Curriculum learning [5] seeks to ease the optimization process by allowing agents to solve progressively more difficult variants of a task.”
BackgroundRobotic Table Tennis with Model-Free Reinforcement LearningPreprint
Gao, W., Graesser, L., Choromański, K., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.14398 · Cited by 5
“We can address these issues using curriculum learning [3] by carefully adjusting two aspects of the problem:”
MethodALLSTEPS: Curriculum-driven Learning of Stepping Stone SkillsPreprint
Xie, Z., Ling, H. Y., Kim, N. H., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04323 · Cited by 5
“Curriculum learning is the learning process where task difficulty is increased overtime during training [BLCW09].”
BackgroundLet's Agree to Agree: Neural Networks Share Classification Order on Real DatasetsPreprint
Hacohen, G., Choshen, L., & Weinshall, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10854 · Cited by 59
“A somewhat related issue, the re-sampling of train points during training based on some notion of point difficulty, has been investigated in the context of curriculum learning (Bengio et al., 2009), self-paced learning (Kumar et al.”
BackgroundMethodDeep Residual Mixture ModelsPreprint
Hämäläinen, P., Trapp, M., Saloheimo, T., & Solin, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12063 · Cited by 9
Suggestive Annotation of Brain Tumour Images with Gradient-guided SamplingPreprint
Dai, C., Wang, S., Mo, Y., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.14984 · Cited by 5
“As revealed in [16,17], less informative samples are more important at the early stage of model training, while harder samples are more important at the later stage.”
BackgroundAdaptive Task Sampling for Meta-LearningPreprint
Liu, C., Wang, Z., Sahoo, D., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08735 · Cited by 5
“Specifically, [5] considers instances that increase difficulty.”
Active Visual Information Gathering for Vision-Language NavigationPreprint
Wang, H., Wang, W., Shu, T., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08037 · Cited by 5
“To avoid this, we adopt curriculum learning[4] to train our agent with incrementally improved exploration length.”
MethodHierarchical Reinforcement Learning in StarCraft II with Human Expertise in Subgoals SelectionPreprint
Xu, X., Huang, T., Wei, P., Narayan, A., & Leong, T.-Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03444 · Cited by 11
Beneficial and Harmful Explanatory Machine LearningPreprint
Ai, L., Muggleton, S., Hocquette, C., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06410 · Cited by 5
“Therefore, learning might be organised incrementally, guided by a curriculum [6,52].”
Background
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.