Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 2,101–2,200 of 5,240
Enhancing Code Language Models for Program Repair by Curricular Fine-tuning Framework
Hao, S., Shi, X., Liu, H., & Shu, Y. (2023) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58846.2023.00024 · Cited by 19
“Bengio et al. [17] are the first to propose curriculum learning strategy and demonstrate its benefits on supervised visual and language learning tasks.”
BackgroundGenetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery
Tarhan, A. F., & Üre, N. K. (2024) · Drones · doi:10.3390/drones8030071 · Cited by 13
Fuentes-Pacheco, J., Román-Rangel, E., Reyes-Rosas, A., Magadán-Salazar, A., Juárez-López, P., Ontiveros‐Capurata, R. E., & Rendón-Mancha, J. M. (2024) · IEEE Access · doi:10.1109/access.2024.3367614 · Cited by 9
“This data first represents simple examples and then contains more complex information as the training progresses [32].”
BackgroundImproved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths
Sun, X., Panda, R., Chen, C.-F. R., Wang, N., Pan, B., Oliva, A., Feris, R., & Saenko, K. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00100 · Cited by 19
“Therefore, we linearly increase the probability to execute the students with respect to the layer depth: Following curriculum learning [4], we set the initial value for p 1 and gradually increase it to 1 during training, i.e. it auto-regresses towards training the network with all student blocks at…”
BackgroundLinguistic Hallucination for Text-Based Video Retrieval
Fang, S., Dang, T., Wang, S., & Huang, Q. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3393843 · Cited by 19
“Data-level CL methods focus on gradually increasing the complexity of data samples during training [55], [56], [57], [58], [59], [60].”
MethodFang, Z., He, L. J., Li, L., & Hu, Y. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3407527 · Cited by 10
“Second, our approach follows the concept of curriculum learning [61], [62], where clean samples are selected from easy to hard and included in the clean set.”
MethodWeighted Self-Paced Learning with Belief Functions
Zhang, S., Han, D., Dezert, J., & Yang, Y. A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124535 · Cited by 13
COMI: COrrect and MItigate Shortcut Learning Behavior in Deep Neural Networks
Zhao, L., Liu, Q., Yue, L., Chen, W., Chen, L., Sun, R., & Song, C. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657729 · Cited by 23
Contrastive graph clustering via enhanced hard sample mining and cluster-guiding
Li, M., Yang, B. Y., Xue, T., Zhang, Y., & Zhou, L. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01567-7 · Cited by 10
“CuCo [40] classifies negative samples in graph contrastive learning by similarity from easy to hard and uses curriculum learning [41] to select and train negative samples automatically.”
Xiao, X., Hu, Q., & Wang, G. (2024) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2024.3516574 · Cited by 11
“Curriculum learning (CL) mechanism: To ensure the adapt-ability of our FgC2F-UDiff for the synthesis of missing data across diverse inputs, we employ curriculum learning (CL) [62] as a rationality-enhancing training strategy.”
Structured Generative Models for Scene Understanding
Williams, C. K. I. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02316-z · Cited by 14
“This allows curriculum learning (Bengio, Louradour, Collobert, & Weston, 2009), where one can first focus on modelling individual object classes using class-conditional data, and then bring in within-scene inter-relationships.”
A Deep Curriculum Learning Semi-Supervised Framework for Remote Sensing Scene Classification
Zhang, Q., Chen, J., & Yuan, B. (2025) · Applied Sciences · doi:10.3390/app15010360 · Cited by 12
Towards the Deployment of Realistic Autonomous Cyber Network Defence: A Systematic Review
Vyas, S., Mavroudis, V., & Burnap, P. (2025) · ACM Computing Surveys · doi:10.1145/3729213 · Cited by 28
“From Cage Challenge 3, Hicks et al. [58] won the challenge by utilising a MARL PPO with curriculum learning [11] to efficiently manage large action spaces, meeting the key requirement of multi-agent collaboration (A.4.1).”
InfluentialAl-Sharman, M., Edes, L., Sun, B., Jayakumar, V., Tahir, H., Daoud, M. A., Emran, B. J., Rayside, D., & Melek, W. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3646982 · Cited by 28
“In practice, convergence of the RL-models in simulation-based environments does not necessarily ensure generalizability in real-life scenarios due to the domains mismatch.”
Webly-Supervised Learning of Multimodal Video Detectors
Liang, J., Jiang, L., & Hauptmann, A. G. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10541 · Cited by 8
“The learning framework is motivated by human learning, where people generally start learning easier aspects of a concept, and then gradually take more complex examples into the learning process(Bengio et al. 2009; Kumar, Packer, and Koller 2010; Jiang et al. 2015a).”
BackgroundLeveraging Multi-modal Prior Knowledge for Large-scale Concept Learning in Noisy Web Data
Liang, J., Jiang, L., Meng, D., & Hauptmann, A. G. (2017) · International Conference on Multimedia Retrieval · doi:10.1145/3078971.3079003 · Cited by 9
Fakoor, R., He, X., Tashev, I., & Zarar, S. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462042 · Cited by 8
“Note that our proposed curriculum learning is different from traditional one [17] such that in the traditional curriculum learning, it first starts with the simpler problem then goes to the harder problem.”
BackgroundMethodInfluentialClyde: A Deep Reinforcement Learning DOOM Playing Agent.
Ratcliffe, D. S., Devlin, S., Kruschwitz, U., & Citi, L. (2017) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · Cited by 9
Speech Activity Detection in Naturalistic Audio Environments: Fearless Steps Apollo Corpus
Kaushik, L., Sangwan, A., & Hansen, J. H. L. (2018) · IEEE Signal Processing Letters · doi:10.1109/lsp.2018.2841653 · Cited by 9
“of the local minima is computed [19].”
DDoS Traffic Control Using Transfer Learning DQN With Structure Information
Xia, S., Zhang, L., Bai, W., Zhou, X., & Pan, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2923993 · Cited by 9
“This idea is known as transfer learning or curriculum learning [10], [18], and it can be extended to RL problems with multiple cooperating agents.”
BackgroundQuantifying Similarity between Relations with Fact Distribution
Chen, W., Zhu, H., Han, X., Liu, Z., & Sun, M. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1278 · Cited by 7
“This can be also viewed as a process of curriculum learning(Bengio et al., 2009), the data fed to the model gradually changes from simple negative triples to hard ones.”
BackgroundFast-PPO: Proximal Policy Optimization with Optimal Baseline Method
Xiao, Z. Q., Xie, N., Yang, G., & Du, Z. (2020) · IEEE International Conference on Progress in Informatics and Computing · doi:10.1109/pic50277.2020.9350833 · Cited by 11
“…deployed A3C with CNN to train an agent in a partially observable 3D environment, Doom, from recent four raw frames and game variables, to predict next action and value function, following the curriculum learning [17] approach of starting with simple tasks and gradual transition to harder ones.”
MethodUğurlu, H. İ., Kalkan, S., & Saranlı, A. (2021) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-021-01412-3 · Cited by 10
“observed that if the training data is presented to the network randomly during supervised learning, it takes longer for the network parameters to converge [4].”
BackgroundImage Data Augmentation techniques for Deep Learning -A Mirror Review
Saini, D., & Malik, R. (2021) · International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) · doi:10.1109/icrito51393.2021.9596262 · Cited by 12
Zhang, H., Luo, J., Lin, X., et al. (2021) · IEEE International Conference on Networking, Sensing and Control (ICNSC) · doi:10.1109/icnsc52481.2021.9702196 · Cited by 9
Better Language Model with Hypernym Class Prediction
Bai, H., Wang, T., Sordoni, A., & Shi, P. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.96 · Cited by 19
Regressing Word and Sentence Embeddings for Low-Resource Neural Machine Translation
Unanue, I. J., Borzeshi, E. Z., & Piccardi, M. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3187680 · Cited by 15
“Finally, curriculum learning has also been explored to use the available training data more effectively [22]–[24].”
MethodYi, M., Zhou, C., Yang, L., et al. (2022) · Entropy · doi:10.3390/e24111696 · Cited by 9
Fisher Information Guidance for Learned Time-of-Flight Imaging
Li, J., Yue, T., Zhao, S., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01585 · Cited by 9
“To train the proposed network in dealing with different noise levels simultaneously, we adopt the incremental training strategy [1].”
MethodModularity in Nervous Systems—a Key to Efficient Adaptivity for Deep Reinforcement Learning
Schilling, M., Hammer, B., Ohl, F. W., Ritter, H. J., & Wiskott, L. (2023) · Cognitive Computation · doi:10.1007/s12559-022-10080-w · Cited by 12
“One direction taken is through broadening the behavioral context, e.g., in curriculum learning [82, 83].”
AtmoDist: Self-supervised representation learning for atmospheric dynamics
Hoffmann, S., & Lessig, C. (2023) · Environmental Data Science · doi:10.1017/eds.2023.1 · Cited by 16
Reinforcement Learning in Few-Shot Scenarios: A Survey
Wang, Z., Fu, Q., Chen, J., Wang, Y., Lu, Y., & Wu, H. (2023) · Journal of Grid Computing · doi:10.1007/s10723-023-09663-0 · Cited by 13
“For dealing with the negative performance after transfer, some works [84] apply Curriculum Learning [85] to adjust the order of learning tasks, and some other works [86] give the guarantee of non-negative transfer in their algorithm.”
MethodCurriculum Learning Based Multi-Agent Path Finding for Complex Environments
Zhao, C., Zhuang, L., Huang, Y., & Liu, H. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191932 · Cited by 9
“In order to have better learning efficiency in complex environments, our CPL is based on the curriculum learning framework [15] in which agents start with simple single-agent path-finding skills and then progressively learn cooperative strategies by network parameter inheritance.”
MethodMueller, A., & Linzen, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.629 · Cited by 45
DynaShare: Task and Instance Conditioned Parameter Sharing for Multi-Task Learning
Rahimian, E., Javadi, G., Tung, F., & Oliveira, G. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00477 · Cited by 14
“Curriculum learning [4] is used to encourage a better convergence.”
MethodEfficient Real-Time Recognition Model of Plant Diseases for Low-Power Consumption Platform
Deng, S., Wu, W., Zou, K., et al. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3307662 · Cited by 9
“Restrictions apply. strategies and accelerate the speed of convergence [58], [59], [60].”
BackgroundOzcelik, M. B., Agin, B., Çaldıran, O., & Sirin, O. (2023) · Innovations in Intelligent Systems and Applications Conference (ASYU) · doi:10.1109/asyu58738.2023.10296611 · Cited by 10
“Additionally, CL method is a widely used training technique for machine learning models [11].”
MethodEo, J., Lee, D., & Kwon, M. (2023) · IEEE Communications Letters · doi:10.1109/lcomm.2023.3339478 · Cited by 10
“Despite its capability, the conventional RL approach, also known as online RL, requires the agent to actively collect trajectory data and undergo trials and errors to learn an optimal policy [13], [14].”
BackgroundMethodA Curriculum Learning Approach to Optimization with Application to Downlink Beamforming
Johnston, J., Liu, X.-Y., Wu, S., & Wang, X. (2023) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2023.3334396 · Cited by 16
“The sequence of tasks should increase in diffi-culty [8], in that the loss functions should become successively more complex, and the entropy of the data distributions should increase as the curriculum progresses.”
BackgroundMethodInfluentialOu, F.-Z., Chen, B., Li, C., Wang, S., & Kwong, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01894 · Cited by 17
“Furthermore, inspired by the curriculum learning strategy [5], several curriculum domain adaptation schemes have been developed in recent years.”
BackgroundCIRI: Curricular Inactivation for Residue-aware One-shot Video Inpainting
Zheng, W., Xu, C., Xu, X., Liu, W., & He, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01196 · Cited by 15
“This scheme shares a similar spirit to curriculum learning [2, 32, 46, 12], which allows the model to start with learning simple tasks and then gradually transfer to difficult tasks for better optimization.”
MethodGeneral Hyperspectral Image Super-Resolution via Meta-Transfer Learning
Cheng, Y., Wang, X., Ma, Y., Mei, X., Wu, M., & Ma, J. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3387970 · Cited by 10
“Furthermore, curriculum learning [8] and hard negative mining [9] both suggest that faster convergence and stronger performance can be achieved by a better arrangement of training data.”
Zhang, S., & Xie, L. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106457 · Cited by 13
FISTNet: FusIon of STyle-path generative Networks for facial style transfer
Khowaja, S. A., Nkenyereye, L., Mujtaba, G., Lee, I. H., Fortino, G., & Dev, K. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102572 · Cited by 12
OTAMatch: Optimal Transport Assignment With PseudoNCE for Semi-Supervised Learning
Zhang, J., Liu, J., Li, D., Huang, Q., Chen, J., & Huang, D. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3425174 · Cited by 13
“The second log-likelihood ratio item log e k e t establishes the divergence between e k and ˆ e t as modification term and is updated at each iteration in the curriculum learning scheme [60], [61].”
BackgroundDan, J., Liu, W., Liu, M., Xie, C., Dong, S., Ma, G., Tan, Y., & Xing, J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680765 · Cited by 12
“To address this issue, motivated by curriculum learning [2], we propose an Adaptive Weighted Domain Alignment ( AWDA ) strategy, which dynamically adjusts the weight of each node based on its transferability.”
MethodConcept-aware embedding for logical query reasoning over knowledge graphs
Pan, P., Lei, J., Wang, J., Ouyang, D., Qu, J., & Li, Z. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103971 · Cited by 11
Spotting Spurious Data with Neural Networks
Amiri, H., Miller, T. A., & Savova, G. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1182 · Cited by 22
“The difference between the above models is in their views to learning: curriculum learning is inspired by the learning principle that training starts with easier concepts and gradually proceeds with more diffi-cult ones (Bengio et al., 2009).”
BackgroundMethodInfluentialLow-Shot Learning From Imaginary 3D Model
Pahde, F., Puscas, M., Wolff, J., Klein, T., Sebe, N., & Nabi, M. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00109 · Cited by 10
“[1] first proposed a general learning strategy: curriculum learning.”
BackgroundPhotorealistic Face Completion with Semantic Parsing and Face Identity-Preserving Features
Ma, R., Hu, H., Wang, W., Xu, J. D., & Li, Z. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3300940 · Cited by 9
“We follow the curriculum strategy [6] and use Adam [18] for optimization.”
MethodMulti-Task Learning with Knowledge Transfer for Facial Attribute Classification
Fanhe, X., Guo, J., Huang, Z., et al. (2019) · IEEE International Conference on Industrial Technology (ICIT) · doi:10.1109/icit.2019.8755180 · Cited by 9
“also improves performance in machine learning [9].”
BackgroundMethodMaking Deep Neural Networks Robust to Label Noise: Cross-Training With a Novel Loss Function
Qin, Z., Zhang, Z., Li, Y., & Guo, J. H. (2019) · IEEE Access · doi:10.1109/access.2019.2940653 · Cited by 13
Knowledge Distillation for End-to-End Monaural Multi-Talker ASR System
Zhang, W., Chang, X., & Qian, Y. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-3192 · Cited by 9
“Moreover, we adopted the curriculum learning [27, 28] technique to further improve the performance.”
BackgroundMethodDeep Reinforcement Learning for Single-Shot Diagnosis and Adaptation in Damaged Robots
Verma, S., Nair, H., Agarwal, G., Dhar, J., & Shukla, A. (2020) · Joint International Conference on Data Science & Management of Data (CODS-COMAD) · doi:10.1145/3371158.3371168 · Cited by 11
“For the Hexapod environment, we also use the concept of curriculum learning [Bengio et al. 2009], by progressively training on cases which are more difficult.”
MethodReview on Brain Tumor Segmentation Methods using Convolution Neural Network for MRI Images
Venu, K., Natesan, P., Sasipriyaa, N., & Poorani, S. (2018) · International Conference on Intelligent Computing and Communication for Smart World (I2C2SW) · doi:10.1109/i2c2sw45816.2018.8997387 · Cited by 10
“Curriculum learning – based training [20] strategy was used to train the multitask model effectively which improves the convergence quality.”
MethodA Traffic Signal Recognition Algorithm Based on Self-paced Learning and Deep Learning
Wang, T., Shen, H., Xue, Y., & Hu, Z. (2020) · Ingénierie des systèmes d information · doi:10.18280/isi.250211 · Cited by 10
“In 2009, Bengio et al. [8] introduced the concept of the CL into the ML, creating the CL training method.”
MethodLearning to Learn Parameterized Classification Networks for Scalable Input Images
Li, D., Yao, A., & Chen, Q. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58526-6_2 · Cited by 11
“Inspired by curriculum learning [3], RCO [15] promotes the student network to mimic the entire route sequence that the teacher network passed by.”
A Curriculum Learning Based Approach to Captioning Ultrasound Images
Alsharid, M., El-Bouri, R., Sharma, H., Drukker, L., Papageorghiou, A. T., & Noble, J. A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60334-2_8 · Cited by 14
“It has been hypothesized that curriculum learning performs a similar function to numerical continuation methods [3], whereby a complex surface is approximated by a smooth version before increasing the complexity of that surface to become more similar to the original [1].”
BackgroundInfluentialA Framework for DRL Navigation With State Transition Checking and Velocity Increment Scheduling
Dewa, C. K., & Miura, J. (2020) · IEEE Access · doi:10.1109/access.2020.3033016 · Cited by 10
“Bengio et al. [35] introduces curriculum learning as a training strategy in the context of machine learning which presents organized examples based on meaningful order, gradually from simple to complex.”
BackgroundActive Pushing for Better Grasping in Dense Clutter with Deep Reinforcement Learning
Lu, N., Lu, T., Cai, Y., & Wang, S. (2020) · Chinese Automation Congress · doi:10.1109/cac51589.2020.9327270 · Cited by 10
“In order to solve this problem, we use the curriculum learning method [30].”
MethodA New Pooling Approach Based on Zeckendorf’s Theorem for Texture Transfer Information
Vigneron, V., Maaref, H., & Syed, T. (2021) · Entropy · doi:10.3390/e23030279 · Cited by 9
“Curricular learning was first proposed by Bengio [37] to deal with nonconvex optimization to avoid the local optimum issue.”
BackgroundNguyen, S. M., Duminy, N., Manoury, A., Duhaut, D., & Buche, C. (2021) · Künstliche Intell. · doi:10.1007/s13218-021-00708-8 · Cited by 15
Modal decomposition of complex optical fields using convolutional neural networks
Schiworski, M. G., Brown, D., & Ottaway, D. J. (2021) · Journal of the Optical Society of America A · doi:10.1364/josaa.428214 · Cited by 10
Efficient Hyperparameter Optimization for Physics-based Character Animation
Yang, Z., & Yin, Z. (2021) · Proceedings of the ACM on Computer Graphics and Interactive Techniques · doi:10.1145/3451254 · Cited by 15
“Curriculum learning (CL) is a learning method where task difficulty gradually increases during training[Bengio et al. 2009].”
BackgroundMethodRobotic Arm Motion Planning Based on Curriculum Reinforcement Learning
Zhou, D., Jia, R., & Yao, H. (2021) · International Conference on Control and Robotics Engineering (ICCRE) · doi:10.1109/iccre51898.2021.9435700 · Cited by 10
“M e t h o d o l o g y : C u r r ic u l u m L e a r n in g M o t io n P l a n n in g Scholar Bengio first proposed the concept of curriculum learning[19] in 2009.”
BackgroundCurriculum Learning for Vehicle Lateral Stability Estimations
Bae, J.-H., Kim, T., Lee, W., & Shim, I. (2021) · IEEE Access · doi:10.1109/access.2021.3090907 · Cited by 9
Tsalakanidou, F., Papadopoulos, S., Mezaris, V., Kompatsiaris, I. Y., Gray, B., Tsabouraki, D., Kalogerini, M., Negro, F., Montagnuolo, M., de Vos, J., et al. (2021) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-79150-6_7 · Cited by 12
“…for unsupervised manifold learning to exploit non-annotated data; trans-4 fer learning to allow reusing models trained for different tasks [7]; transfer neural architecture search to find suitable DNN architectures fast by leveraging past experience in different datasets [8]; and deep…”
Low-level Pose Control of Tilting Multirotor for Wall Perching Tasks Using Reinforcement Learning
Lee, H., Jeong, M., Kim, C., Lim, H., Park, C., Hwang, S., & Myung, H. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636372 · Cited by 10
Radiomics-Informed Deep Curriculum Learning for Breast Cancer Diagnosis
Nebbia, G., Dadsetan, S., Arefan, D., Zuley, M. L., Sumkin, J. H., Huang, H., & Wu, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87240-3_61 · Cited by 16
Rao, J., Qian, T., Qi, S., Wu, Y., Liao, Q., & Wang, X. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482194 · Cited by 10
“Inspired by these[1, 4, 14, 18] works, we propose to extract and compress knowledge on the cross-modal pre-train model using a cross-modal model compression method, termed as time-dependent module replacement(TDMR).”
MethodReliable shot identification for complex event detection via visual-semantic embedding
Luo, M., Chang, X., & Gong, C. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103300 · Cited by 10
“Since the instances with low reliability are difficult to use for training a robust classifier, we are motivated by curriculum learning (Bengio et al., 2009) and start training the classifier on high-reliability instances first, and then gradually take the instances with relatively low reliability into…”
BackgroundMulti-agent Collaborative Learning with Relational Graph Reasoning in Adversarial Environments
Wu, S., Qiu, T., Pu, Z., & Yi, J. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636636 · Cited by 9
“In addition, to speed up the training, curriculum learning [24] based on model load is adopted to train the policy from a small number of agents to a large number of agents.”
BackgroundDong, H., Zhang, H., Ding, F., et al. (2021) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2021.3138140 · Cited by 9
End-to-end learning for off-road terrain navigation using the Chrono open-source simulation platform
Benatti, S., Young, A., Elmquist, A., Taves, J., Tasora, A., Serban, R., & Negruţ, D. (2022) · Multibody System Dynamics · doi:10.1007/s11044-022-09816-1 · Cited by 16
“We adopted a curriculum learning approach [41], progressively increasing the complexity of the task as shown in Fig.”
MethodLearning a family of motor skills from a single motion clip
Lee, S., Lee, S., Lee, Y.-W., et al. (2021) · ACM Transactions on Graphics · doi:10.1145/3476576.3476652 · Cited by 9
“A popular approach to overcome this problem is to use curriculum learning [Bengio et al. 2009].”
MethodSemantic Image Segmentation Using Scant Pixel Annotations
Chakravarthy, A. D., Abeyrathna, D. L. B., Subramaniam, M., Chundi, P., & Gadhamshetty, V. R. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4030029 · Cited by 10
Ng, K. W., Furqan, M. S., Gao, Y., Ngiam, K. Y., & Khoo, E. T. (2023) · Electronics · doi:10.3390/electronics12020292 · Cited by 8
Rethinking Semi-supervised Learning with Language Models
Shi, Z., Tonolini, F., Αλέτρας, Ν., Yılmaz, E., Kazai, G., & Jiao, Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.347 · Cited by 36
Abrupt and spontaneous strategy switches emerge in simple regularised neural networks
Löwe, A. T., Touzo, L., Muhle-Karbe, P. S., Saxe, A., Summerfield, C., & Schuck, N. W. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012505 · Cited by 16
“Finally, previous work has also found substantial individual differences between neural network instances that are induced by random differences in weight initialisation, noise, or the order of training examples [38, 39], which can become larger with training [40].”
BackgroundScalable Lifelong Learning with Active Task Selection
Ruvolo, P., & Eaton, E. (2013) · Cited by 9
Behavioral Factors in Interactive Training of Text Classifiers
Settles, B., & Zhu, X. (2012) · Cited by 9
Self-paced model learning for robust visual tracking
Huang, W., Gu, J. J., Ma, X., & Li, Y. (2017) · Journal of Electronic Imaging · doi:10.1117/1.jei.26.1.013016 · Cited by 9
N-RPN: Hard Example Learning For Region Proposal Networks
Cho, M., Chung, T.-Y., Lee, H., & Lee, S. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803519 · Cited by 11
“[10] shows that start learning with easier task and then gradually increasing the difficulty of learning can improve the generalization and faster convergence, we first train RPN without nRPN.”
BackgroundBiSPL: Bidirectional Self-Paced Learning for Recognition From Web Data
Wu, X., Chang, J., Lai, Y., Yang, J., & Tian, Q. C. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3094744 · Cited by 9
“Inspired by the learning pattern of humans, curriculum learning (CL) [7], [51], [52] gradually incorporates training samples into several ordered curriculums from easy to hard.”
BackgroundMethodGouws, S., & G-J, V. R. (2012) · SUNScholar (Stellenbosch University) · Cited by 7
Deictic Image Mapping: An Abstraction for Learning Pose Invariant Manipulation Policies
Platt, R., Kohler, C., & Gualtieri, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018042 · Cited by 11
“Curriculum learning: Another thing that can speed up training is curriculum learning (Bengio et al. 2009).”
BackgroundContext-Dependent Translation Selection Using Convolutional Neural NetworkPreprint
Tu, Z., Hu, B., Lu, Z., & Li, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.02357 · Cited by 9
“It consists of two components: • convolutional sentence model that summarizes the meaning of the source sentence and the target phrase; • matching model that compares the two representations with a multi-layer perceptron (Bengio, 2009).”
BackgroundAn Extensive Evaluation of Filtering Misclassified Instances in Supervised Classification TasksPreprint
Smith, M. R., & Martinez, T. R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.3970 · Cited by 12
Cross Quality Distillation.Preprint
Su, J.-C., & Maji, S. (2016) · arXiv (Cornell University) · Cited by 9
Information Theoretically Aided Reinforcement Learning for Embodied AgentsPreprint
Montúfar, G., Ghazi-Zahedi, K., & Ay, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09735 · Cited by 12
“Curriculum learning (Bengio et al., 2009) and transfer learning (Pan and Yang, 2010) can be regarded as examples of this general strategy.”
BackgroundIncremental Sequence LearningPreprint
de Jong, E. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03068 · Cited by 19
“Increasing training set size: following (Bengio et al., 2009), which describes curriculum sequence learning, we use subsets of the training data that grow in size.”
MethodInfluentialAnnealed Generative Adversarial NetworksPreprint
Mehrjou, A., Schölkopf, B., & Saremi, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.07505 · Cited by 12
“Finally, one can also motivate β-GAN from the perspective of curriculum learning [3], where learning the uniform distribution is the initial task in the curriculum.”
BackgroundCIFT: Crowd-Informed Fine-Tuning to Improve Machine Learning AbilityPreprint
Lalor, J. P., Wu, H., & Yu, H. Q. (2017) · arXiv (Cornell University) · Cited by 11
Improved training for online end-to-end speech recognition systemsPreprint
Kim, S., Seltzer, M. L., Li, J., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02212 · Cited by 9
“It was proposed to address the challenges of training deep neural networks under non-convex training criteria [23].”
BackgroundPoint in, Box out: Beyond Counting Persons in CrowdsPreprint
Liu, Y., Shi, M., Zhao, Q., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01333 · Cited by 9
“Instead of training the model on the entire set once, we adopt a curriculum learning strategy [3, 38, 48] to train the model from images of relatively accurate and easy pseudo ground truth first.”
BackgroundMethodMulti-modal Discriminative Model for Vision-and-Language NavigationPreprint
Huang, H., Jain, V., Mehta, H., Baldridge, J., & Ie, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13358 · Cited by 18
“To further confirm this hypothesis, we train a discriminator using curriculum learning (Bengio et al., 2009) where the model is first trained on only PS negatives and then fine-tuned on PR and RW negatives.”
MethodDeepLogic: Towards End-to-End Differentiable Logical ReasoningPreprint
Cingillioglu, N., & Russo, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07433 · Cited by 17
“Training We use two training regimes: curriculum learning (Bengio et al. 2009) and multi-task.”
MethodMass Personalization of Deep LearningPreprint
Schneider, J., & Vlachos, M. (2019) · arXiv (Cornell University) · Cited by 12
Controllable and Progressive Image ExtrapolationPreprint
Li, Y., Jiang, L., & Yang, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11711 · Cited by 9
“The line of research work generally regards finding an optimal order of executing some known tasks [3].”
BackgroundHavenstrøm, S. T., Rasheed, A., & San, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09792 · Cited by 9
“• Exploring the potential of curriculum learning (Bengio et al., 2009) in the context of training AUVs.”
Background
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.