Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 3,801–3,900 of 5,240
How Do Humans Teach: On Curriculum Design for Machine Learners
Peng, B. (2017) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/3091125.3091468 · Cited by 1
Single Channel Speech Separation Using Deep Neural Network
Chen, L., Ma, X., & Ding, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-59072-1_34 · Cited by 1
A curriculum-based approach for feature selection
Kalavala, D., & Bhagvati, C. (2017) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2280496 · Cited by 1
Learning and Reasoning With Constraints
Gori, M. (2017) · Elsevier eBooks · doi:10.1016/b978-0-08-100659-7.00006-3 · Cited by 1
Towards Developmental AI: The paradox of Ravenous Intelligent Agents.
Diligenti, M., Gori, M., & Maggini, M. (2011) · Cited by 1
Learning from outside the viability kernel: Why we should build robots that can fall with grace
Heim, S. W., & Badri–Spröwitz, A. (2018) · IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR) · doi:10.1109/simpar.2018.8376271 · Cited by 2
From Easy to Difficult: A Self-Paced Multi-Task Joint Sparse Representation Method
Lihua, G. U. O. (2018) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2017edp7289 · Cited by 1
“[11] Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” Proc.”
BackgroundProgressive Cross-Media Correlation Learning
Huang, X., & Peng, Y. (2018) · Communications in computer and information science · doi:10.1007/978-981-13-1702-6_20 · Cited by 1
Visual Search as a Probabilistic Sequential Decision Process in Software Autonomous System
Ghosh, A., & Huang, S. (2018) · IEEE SoutheastCon · doi:10.1109/secon.2018.8479127 · Cited by 1
“To solve this problem, RL algorithms acquire knowledge in several manners like transfer learning [45], multitask learning [46] and curriculum learning [47] etc.”
BackgroundSSL: Synchronous Self-paced Learning for Internet-of-Things Devices
Bao, Y., Ma, L., & Chen, W. (2018) · doi:10.1145/3277893.3277898 · Cited by 1
“Our work is firstly inspired by the existing automatic model construction methods presented in [4], [9] and [10], and about the sample selection method, our work is inspired by the self-paced learning method presented in [2].”
BackgroundMethodApplying Online Expert Supervision in Deep Actor-Critic Reinforcement Learning
Zhang, J., Chen, J., Huang, Y., Wan, W., & Li, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-03335-4_41 · Cited by 2
Self-Paced Densely Connected Convolutional Neural Network for Visual Tracking
Ge, D., Song, J., Qi, Y., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-03341-5_9 · Cited by 1
Teach to Hash: A Deep Supervised Hashing Framework with Data Selection
Li, X., Ma, C., Yang, J., & Qiao, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-04167-0_11 · Cited by 1
Training Speech Recognition Models on HPC Infrastructure
Karkada, D., & Saletore, V. A. (2018) · doi:10.1109/mlhpc.2018.8638637 · Cited by 2
“These sorted indexed audio samples are then processed in increasing order of lengths to satisfy the terms of curriculum learning algorithm [38].”
BackgroundMAIN: Multi-Attention Instance Network for video segmentation
Alcázar, J. L., Bravo, M. A., Jeanneret, G., Thabet, A., Brox, T., Arbeláez, P., & Ghanem, B. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103240 · Cited by 3
“To adapt MAIN to the error pattern in the tracker, we use a Curriculum Learning [1] strategy.”
MethodLeveraging Simple Model Predictions for Enhancing its Performance
Dhurandhar, A., Shanmugam, K., & Luss, R. (2019) · Cited by 2
Efficient and Robust Learning on Elaborated Gaits with Curriculum Learning
Zhou, B., Zeng, H., Wang, F., Lian, R., & Tian, H. (2019) · The Springer series on challenges in machine learning · doi:10.1007/978-3-030-29135-8_10 · Cited by 2
My Health Sensor, My Classifier – Adapting a Trained Classifier to Unlabeled End-User Data
Nikolaidis, K., Kristiansen, S., Plagemann, T., Goebel, V., Liestøl, K., Kankanhalli, M., Traaen, G. M., Øverland, B., Akre, H., Aakerøy, L., et al. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3559767 · Cited by 1
“At irst glance SICO and Curriculum Learning (CL) [7] can appear to be quite similar.”
BackgroundCBCR: A Curriculum Based Strategy For Chromosome Reconstruction
Hovenga, V., & Oluwadare, O. (2021) · International Journal of Molecular Sciences · doi:10.3390/ijms22084140 · Cited by 1
Multi-contextual design of convolutional neural network for steganalysis
Singh, B., Sur, A., & Mitra, P. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18545-w · Cited by 3
“The solution to this kind of problem has been found using curriculum and transfer learning [43, 44].”
BackgroundStyle Curriculum Learning for Robust Medical Image Segmentation
Liu, Z., Manh, V., Yang, X. S., et al. (2021) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.2108.00402 · Cited by 1
“Curriculum learning methods [1,6,20] can guide the model to learn better by organising the training samples in a meaningful order.”
BackgroundFacial landmark detection transfer learning for a specific user in driver status monitoring systems
Kim, J., Taguchi, K., Hayashi, Y., Miyazaki, J., & Fujiyoshi, H. (2021) · doi:10.23919/mva51890.2021.9511385 · Cited by 1
“Several methods for pseudo labeling of unlabeled data in semi-supervised scenarios have been proposed[19, 7, 3].”
MethodBeyond Full Supervision in Deep Learning
Thome, N. (2012) · Springer eBooks · doi:10.1007/978-3-030-74478-6_4 · Cited by 2
Discrete Task-Space Automatic Curriculum Learning for Robotic Grasping
Kurkcu, A., Acar, C., Campolo, D., & Tee, K. P. (2021) · International Conference on Control, Automation and Systems (ICCAS) · doi:10.23919/iccas52745.2021.9649917 · Cited by 2
Mislabeled Samples Adjustment Based on Self-paced Learning Framework
Huang, Z., Li, X., Deng, L., et al. (2021) · International Conference on Computer and Communications (ICCC) · doi:10.1109/iccc54389.2021.9674334 · Cited by 1
Behaviour of Sample Selection Techniques Under Explicit Regularization
Lakshya, L. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-90439-5_26 · Cited by 1
Curriculum Learning for Age Estimation from Brain MRI
Asan, A., Terzi, R., & Azgınoğlu, N. (2021) · Applied Computer Systems · doi:10.2478/acss-2021-0014 · Cited by 1
Reinforcement Learning and Affective Computing
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502405 · Cited by 1
“This is true even in modern times [Bengio et al. 2009].”
BackgroundExploring Effective Data Utilization for Low-Resource Speech Recognition
Zhou, Z., Wang, W., Zhang, W., & Qian, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747543 · Cited by 5
Does BERT Know Which Answer Beyond the Question?
Xia, F., Weng, Y., Xia, M., Yu, Q., He, S., Liu, K., Liu, S., & Zhao, J. (2022) · Communications in computer and information science · doi:10.1007/978-981-19-0713-5_9 · Cited by 1
A precise method for RBMs training using phased curricula
Wang, Q., Gao, X., Li, X., Hu, Z., & Wan, K. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-12973-2 · Cited by 3
Generative and discriminative infinite restricted Boltzmann machine training
Wang, Q., Gao, X., Wan, K., & Hu, Z. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.22908 · Cited by 2
SEGP: Stance-Emotion Joint Data Augmentation with Gradual Prompt-Tuning for Stance Detection
Wang, J., Zhou, Y., Liu, Y., Zhang, W., & Hu, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08757-8_48 · Cited by 2
“Curriculum learning is proposed by Bengio et al. [4], which is a training strategy that imitates the meaningful learning order in human curricula.”
BackgroundSpatial relation learning in complementary scenarios with deep neural networks
Lee, J. H., Yao, Y., Özdemir, O., Li, M., Weber, C., Liu, Z., & Wermter, S. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2022.844753 · Cited by 2
“For machine learning tasks that span multiple levels of difficulty, curriculum learning has been shown to be efficient for a variety of models (Elman, 1993; Bengio et al., 2009).”
MethodPlanning with Q-Values in Sparse Reward Reinforcement Learning
Lei, H., Weng, P., Rojas, J., & Guan, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13844-7_56 · Cited by 1
Xu, S., Wan, X., Hu, S., Zhou, M., Xu, T., Wang, H., & Mi, H. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539116 · Cited by 2
“In order for the model to perform better, we borrow the idea of curriculum learning [3].”
MethodHong, K., Guo, L., & Lure, Y. M. F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16431-6_65 · Cited by 1
Ensemble Self-Paced Learning Based on Adaptive Mixture Weighting
Liu, L., Wang, Z., Bai, J., Yang, X., Yang, Y., & Zhou, J. (2022) · Electronics · doi:10.3390/electronics11193154 · Cited by 1
Self-aware cycle curriculum learning for multiple-choice reading comprehension
Chen, H., & Li, Y. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.1179 · Cited by 1
Kaushik, P., Kumar, K. V., & Biswas, P. (2022) · 2022 8th International Conference on Signal Processing and Communication (ICSC) · doi:10.1109/icsc56524.2022.10009634 · Cited by 2
Motion-Primitive based Deep Reinforcement Learning for High Speed Aerospace Vehicle Missions
Levin, W. C., Nolan, S. M., Raz, A. K., Ezra, K., Parish, J. J., & Williams, K. (2023) · AIAA SciTech Forum · doi:10.2514/6.2023-2667 · Cited by 2
Efficient Inference on Convolutional Neural Networks by Image Difficulty Prediction
Chang, Y.-J., Hong, D., Liu, P., & Wu, J. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020754 · Cited by 1
“Bengio et al. [21] first proposed this concept and showed that curriculum learning leads to faster convergence and lower error rates.”
BackgroundReward estimation with scheduled knowledge distillation for dialogue policy learning
Qiu, J., Zhang, H., & Yang, Y. (2023) · Connection Science · doi:10.1080/09540091.2023.2174078 · Cited by 2
“Curriculum learning (Bengio et al., 2009) (CL) is a strategy that trains a machine learning model by learning from easy samples and gradually increasing the complexity.”
BackgroundVoltage Regulation of DC-DC Buck Converters Feeding CPLs via Automatic Curriclum Learning
Min, Z. X., Gang, C. C., & Xiao, Y. T. (2022) · International Conference on Electrical Engineering and Control Technologies (CEECT) · doi:10.1109/ceect55960.2022.10030117 · Cited by 2
“So, people generally tend to present "easy" data first and then gradually give "difficult" data [10], The convergence and prediction performance of the algorithm can be significantly improved by the step difficulty pre-partitioning of the data.”
BackgroundEnd to End Optimized Tiny Learning for Repositionable Walls in Maze Topologies
Pau, D. P., Colella, S., & Marchisio, C. (2023) · International Conference on Consumer Electronics · doi:10.1109/icce56470.2023.10043558 · Cited by 1
“This approach had some conceptual analogies with Curriculum Learning (CL) [23] such as the intention to present to the agent episodes that are not randomly chosen but following a criterion.”
MethodDomain Word Extension Using Curriculum Learning
Seong, S., & Cha, J.-W. (2023) · Sensors · doi:10.3390/s23063064 · Cited by 2
“The curriculum learning [13] is a learning strategy formulated in the context of machine learning that humans and animals learn better when training in the order”
BackgroundNoise-Based and Class-Based Curriculum Learning for Image Classifiers
Li, Y., & Urtāns, Ē. (2023) · International Conference on Computer Research and Development · doi:10.1109/iccrd56364.2023.10080241 · Cited by 1
“This idea has been introduced to the field of machine learning, and the term "curriculum learning" was coined in the paper [1].”
BackgroundDomestic Large Model Technology and Medical Applications Analysis
Zhang, P. C., & Yu, M. X. (2023) · Advanced ultrasound in diagnosis and therapy · doi:10.37015/audt.2023.230027 · Cited by 2
Temporal Contrastive Learning with Curriculum
Roy, S., & Etemad, A. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096811 · Cited by 4
“Curriculum learning [8] is a paradigm that is inspired by the human learning behaviour of staring with ‘easy’ concepts and slowly learning more complex topics.”
BackgroundActivity-Informed Industrial Audio Anomaly Detection Via Source Separation
Kim, J., Lee, Y., Cho, H. M., Kim, D. W., Song, C. H., & Ok, J. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095113 · Cited by 1
“Indeed, given independent random variables Y 1 and Y 2 , the entropy of mixture X = Y 1 + Y 2 is higher than that of individual random variable, i.e., The entropy is empirically interpreted as a measure of learning difficulty [17, 18], and less difficult data can lead easier anomaly detection [19].”
BackgroundHeuristic search of optimal machine teaching curricula
Piqueras, M. G., & Hernández‐Orallo, J. (2023) · Machine Learning · doi:10.1007/s10994-023-06347-4 · Cited by 2
“Bengio et al. (2009) initiated the formalisation of curriculum learning (CL) in the context of machine learning.”
BackgroundToward More Robust Classifier: Negative Log-Likelihood Aware Curriculum Learning
Kar, I., Chatterjee, A. S., Mukhopadhyay, S., et al. (2023) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-99-0609-3_8 · Cited by 1
Curriculum-Based Imitation of Versatile Skills
Li, M. X., Celik, O., Becker, P., Blessing, D., Lioutikov, R., & Neumann, G. M. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160543 · Cited by 4
“While curricula are a well-studied technique to improve the performance of supervised [20, 21] and reinforcement learning [22] approaches, we are not aware of any approaches using curricula for skill-based imitation learning.”
BackgroundASPER: Answer Set Programming Enhanced Neural Network Models for Joint Entity-Relation Extraction
Le, T., Cao, H., & Son, T. C. (2023) · Theory and Practice of Logic Programming · doi:10.1017/s1471068423000297 · Cited by 2
Terminology-Enriched Meta-curriculum Learning for Domain Neural Machine Translation
Chen, Z., & Wang, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-4752-2_32 · Cited by 1
Code Vulnerability Detection via Signal-Aware Learning
Suneja, S., Zhuang, Y., Zheng, Y., Laredo, J., Morari, A., & Khurana, U. (2023) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp57164.2023.00037 · Cited by 3
“We approach model signal-awareness enhancement by marrying the SE concept of code complexity with the ML technique of curriculum learning [31].”
BackgroundMethodLearning Cooperative Policies with Graph Networks in Distributed Swarm Systems
Zhang, T., Liu, Z., Pu, Z., Yi, J., Ai, X., & Yuan, W. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191136 · Cited by 2
“Besides, curriculum learning [23] is adopted to speed up the process of learning cooperative policies.”
BackgroundCurating Datasets for Better Performance with Example Training Dynamics
Sar-Shalom, A., & Schwartz, R. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.674 · Cited by 1
Improving Autoregressive Grammatical Error Correction with Non-autoregressive Models
Cao, H., Cao, Z., Hu, C., Hou, B., Xiao, T., & Zhu, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.760 · Cited by 2
“Inspired by curriculum learning (Bengio et al., 2009), which is to imitate the human learning process, we propose the dynamic weight strategy.”
MethodAutomated Software Vulnerability Detection via Curriculum Learning
Du, Q., Kun, W., Kuang, X., et al. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00485 · Cited by 1
“Similar to the human learning process accompanied by an easy-tohard curriculum arrangement, some research works [13]–[15] show that presenting training examples in a meaningful order (starting from easy ones and gradually moving on to difficult ones), is beneficial to the learning process.”
ResultMoreau, J., Mechtouff, L., Rousseau, D., et al. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230836 · Cited by 1
“Initially, vanilla CL [6] was defined as follows: more difficult examples are added over time based on human-defined criteria.”
BackgroundSmart suspect tracker using deep learning model
Gotlur, K., Patham, G., Pallati, L., et al. (2023) · AIP conference proceedings · doi:10.1063/5.0161992 · Cited by 1
Curriculum Knowledge Switching for Pancreas Segmentation
Tang, Y., Zhan, K., Tian, Z., Zhang, M., Wang, S., & Wen, X. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10222027 · Cited by 1
“According to this observation, we propose a novel Curriculum Knowledge Switching framework inspired by curriculum learning [5].”
MethodThe Emergence of Cooperation in Multi-Agent Pursuit-Evasion Game
Wei, B., Li, Z., Zhang, X., et al. (2023) · Chinese Control Conference (CCC) · doi:10.23919/ccc58697.2023.10240000 · Cited by 1
“In this paper, we combine MADDPG with curriculum learning (CL) [13], in which agent learns in an easy environment first and gradually advances to difficult environment.”
Addressing Long-Distance Dependencies in AMR Parsing with Hierarchical Clause Annotation
Fan, Y., Li, B., Sataer, Y., Gao, M., Shi, C., & Gao, Z. (2023) · Electronics · doi:10.3390/electronics12183908 · Cited by 1
Find Important Training Dataset by Observing the Training Sequence Similarity
Liu, Z., Diao, H., Zhang, F., & Khan, S. U. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44213-1_34 · Cited by 1
Automatic Action Space Curriculum Learning with Dynamic Per-Step Masking
Yasutomi, A. Y., & Ogata, T. (2023) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case56687.2023.10260397 · Cited by 4
“While traditional CL approaches organize tasks and environments (training data) by order of complexity to progressively introduce them to the training agent to improve learning [5], action-space CL approaches progressively change the”
BackgroundEvolutionary Machine Learning and Games
Togelius, J., Khalifa, A., Earle, S., Green, M. C., & Soros, L. B. (2023) · Genetic and evolutionary computation · doi:10.1007/978-981-99-3814-8_25 · Cited by 3
Towards Goal-Oriented Agents for Evolving Problems Observed via Conversation
Free, M., Langworthy, A., Dimitropoulaki, M., & Thompson, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47994-6_11 · Cited by 1
“During testing, we experimented with several architectures for the RL agent.”
MethodGray Learning From Non-IID Data With Out-of-Distribution Samples
Zhao, Z., Cao, L., & Wang, C. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3330475 · Cited by 2
“Lastly, our GL method also involves Curriculum Learning [10] and Negative Learning [21].”
BackgroundMethodEmpowering Wearable Seizure Forecasting with Scheduled Sampling
Guo, P., Yu, H., Karicheri, S. G., Kuncheria, A., Yang, H., Blackwell, S., Haneef, Z., & Sano, A. (2023) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi58575.2023.10313368 · Cited by 1
“This strategy of progressively introducing more complex tasks to the model is analogous to human learning [23].”
BackgroundElicitation-Based Curriculum Learning for Improving Speech Emotion Recognition
Kumawat, P., Routray, A., & Chaudhuri, S. P. (2023) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon51785.2023.10311721 · Cited by 1
“This learning is usually known as curriculum learning (CL) [15].”
BackgroundMethodInfluentialSarkar, T., & Kalita, S. (2023) · SN Computer Science · doi:10.1007/s42979-023-02326-7 · Cited by 2
GPT-wee: How Small Can a Small Language Model Really Get?
Bunzeck, B., & Zarrieß, S. (2023) · BabyLM Challenge · doi:10.18653/v1/2023.conll-babylm.2 · Cited by 7
“Curriculum learning is an approach to machine learning where “the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones” (Bengio et al., 2009, 41).”
BackgroundLing-CL: Understanding NLP Models through Linguistic Curricula
Elgaar, M., & Amiri, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.834 · Cited by 3
Complexity-Guided Curriculum Learning for Text Graphs
Vakil, N., & Amiri, H. (2023) · doi:10.18653/v1/2023.findings-emnlp.172 · Cited by 1
Boumendil, A., Bechkit, W., & Benatchba, K. (2023) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai59109.2023.00054 · Cited by 2
“A dynamic size of the training set size has been explored in parallel work on curriculum learning, where examples are ordered according to their difficulty [43].”
BackgroundChung, S.-C. (2023) · Journal of Structural Biology · doi:10.1016/j.jsb.2023.108058 · Cited by 1
Reinforcement Learning for Turn-Based Strategy Game
Jonathan, G., & Maulidevi, N. U. (2023) · International Conference on Advanced Informatics: Concepts, Theory and Applications (ICAICTA) · doi:10.1109/icaicta59291.2023.10390086 · Cited by 1
Graph-Based Autonomous Driving with Traffic-Rule-Enhanced Curriculum Learning
Peiss, L. F., Wohlgemuth, E., Xue, F., Meyer, E., Gressenbuch, L., & Althoff, M. (2023) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc57777.2023.10422140 · Cited by 7
“CL schemes subdivide the training process of machine learning algorithms into a sequence of sub-tasks of increasing difficulty [4].”
BackgroundChoi, M.-Y., Shin, W., Kim, M., Park, H.-M., You, Y., Lee, M., & Oh, H. (2024) · The Journal of Korea Robotics Society · doi:10.7746/jkros.2024.19.1.117 · Cited by 1
Learning Spiking Neural Network from Easy to Hard Task
Tang, L., Hu, J., Hua, Y., Liu, S., & Chu, J. (2023) · International Conference on Digital Society and Intelligent Systems (DSInS) · doi:10.1109/dsins60115.2023.10455649 · Cited by 1
“Its main idea is to gradually increase the training difficulty, allowing the model to learn useful knowledge more quickly and better generalize to new data in later stages of training [13].”
BackgroundSequentia12D: Organizing Center of Skip Connections for Transformers
Pathak, H. N., Paffenroth, R. C., & Hershey, Q. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00057 · Cited by 1
Reinforcement Learning Based End-to-end Autonomous Obstacle Avoidance for Manipulators
Zhang, W., Chen, D., Sun, X., et al. (2023) · China Automation Congress (CAC) · doi:10.1109/cac59555.2023.10451444 · Cited by 1
Feyisa, D. W., Ayano, Y. M., Debelee, T. G., & Hailu, S. S. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-57624-9_2 · Cited by 1
MATSUO, H., Sato, N., & Morita, Y. (2023) · IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) · doi:10.1109/ssrr59696.2023.10499955 · Cited by 1
“The concept of curriculum learning was introduced to help agents accomplish difficult tasks such as this test.. Curriculum learning is a method in which tasks are made easy in the early stages of learning and more difficult as the learning progresses [10].”
MethodPatel, A., & Gloor, P. A. (2023) · IEEE MIT Undergraduate Research Technology Conference (URTC) · doi:10.1109/urtc60662.2023.10534950 · Cited by 1
“Curriculum learning (CL) is a technique proposed by Bengio et al. to improve reinforcement learning training times [5].”
MethodGrahn, D., Chen, L., & Zhang, J. (2023) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom60117.2023.00219 · Cited by 1
Shi, Z., & Xiong, B. (2024) · International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) · doi:10.1109/ainit61980.2024.10581585 · Cited by 2
“In this study, we trained based on the stable diffusion model and proposed an innovative curriculum learning [31] fine-tuning method aimed at improving the performance and generalization ability of the model on the task of text-to-image generation of Yao ethnic clothing.”
MethodTowards a Process Algebra and Operator Theory for Learning System Objects
Cody, T., & Beling, P. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-65572-2_5 · Cited by 1
Liu, Q., King, A., & Ge, T. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00208 · Cited by 1
“The basic idea of curriculum learning (CL) [50] is to sort examples into a sequence ordered by easiness.”
BackgroundExploring Continual Learning of Compositional Generalization in NLI
Fu, X., & Frank, A. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00680 · Cited by 10
“…the order of continual learning by observing dependencies and degrees of difficulty of primitive and compositional inference types, for compositional NLI performance. v) We control for data leakage using pseudo data and demonstrate that the C 2 Gen NLI challenge persists for LLMs such as Llama.”
MethodKong, K., Kim, K., & Kang, S. (2024) · IEEE Access · doi:10.1109/access.2024.3439561 · Cited by 2
Continual Driving Policy Optimization with Closed-Loop Individualized Curricula
Niu, H., Xu, Y., Jiang, X., & Hu, J. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611578 · Cited by 9
“In addition to its original definition [30], CL has been extended to a series of similar or expanded algorithms, such as self-paced learning (SPL) [31]–[34] and teacher-guided learning [35]– [39].”
BackgroundEmara, S., Liu, D., Fei, W., et al. (2024) · IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) · doi:10.1109/infocomwkshps61880.2024.10620892 · Cited by 1
““easy” experiences are supposed to be not “too easy” to be useless to learning, or not “too difficult” [10].”
BackgroundInformed Machine Learning for Cardiomegaly Detection in Chest X-Rays: A Comparative Study
Hasse, F. C., Leiser, F., & Sunyaev, A. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635719 · Cited by 1
“The second approach is curriculum learning where models are trained on easier examples first [8].”
BackgroundMethodKarimi, F., Mehrpanah, A., & Rawassizadeh, R. (2024) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2024.104784 · Cited by 4
Large Language Model with Curriculum Reasoning for Visual Concept Recognition
Zhang, Y. J., Wang, X. Y., Chen, H., Fan, J., Wen, W., Xue, H., Mei, H., & Zhu, W. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671653 · Cited by 14
“Inspired by human learning processes, curriculum learning attempts to train deep neural networks by starting with the simple samples and gradually increasing the difficulty of the data samples [3, 37, 54, 57, 71].”
BackgroundSmaller Can Be Better: Efficient Data Selection for Pre-training Models
Fang, G., Wang, S., Wang, M., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-7232-2_22 · Cited by 1
End-to-End Fluorescence Lifetime Imaging with Optimized Encoding and Exposure Allocation
Guo, Z., Li, J., Wang, K., Yue, T., & Hu, X. (2024) · IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccp61108.2024.10644727 · Cited by 1
“To enhance the adaptability of the coding scheme and the reconstruction network to different SNRs caused by different exposure times (e.g. the shorter total exposure time corresponds to a lower SNR), we adopt a curriculum learning [45] approach for training mixed exposure times.”
Method
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.