Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 4,401–4,500 of 5,240
Vision Uformer: Long-Range Monocular Absolute Depth EstimationPreprint
Polasek, T., Čadík, M., Keller, Y., et al. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4184429
Hu, S., Ye, Q., Li, F., et al. (2022) · Research Square · doi:10.21203/rs.3.rs-1613749/v1
ECL: Exclusive Curriculum Learning for Video Super-Resolution
Hu, S., Wang, Z., Yi, P., He, Z., Xiao, J., & Xiao, J. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859754
“Curriculum learning [12] is inspired from the human learning rules that humans tend to learn from the easy cases to difficult ones.”
BackgroundMUST Augment: Efficient Augmentation with Multi-stage Stochastic Strategy
Li, Q., Xie, S., Oymagil, A., Zhang, Z., Eseoğlu, M. F., & Lee, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-15919-0_24
Learning High-Risk High-Precision Motion Control
Kim, N. H., Kirjonen, M., & Hämäläinen, P. (2022) · Motion in Games · doi:10.1145/3561975.3562943
“To prevent such failure modes, we implement curriculum learning [Bengio et al. 2009] with the following three stages: (1) In the first stage (Figure 7a), only the action means are learned while the action variance and the component weights are held constant.”
Building Machine Learning Bot with ML-Agents in Tank Battle
Dung, V. D., & Hung, P. D. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-16865-9_10
Attentive Reinforcement Learning for Scheduling Problem with Node Auto-scaling
Guan, Y., Xie, M., Li, Y., & Xu, X. (2022) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53654.2022.9945288
“Curriculum learning is first used in the supervised learning [30] and then gradually used extensively in various fields, especially in the area of reinforcement learning [31]–[33].”
BackgroundNie, X., Liang, Y., Han, Z., & Ohkura, K. (2022) · Artificial Life and Robotics · doi:10.1007/s10015-022-00833-z
“A promising approach to solve this problem is curriculum learning (CL) [10].”
BackgroundA Curriculum Batching Strategy for Automatic ICD Coding with Deep Multi-Label Classification Models
Wang, Y., Han, X., Hao, X., Zhu, T., & Shu, H. (2022) · Healthcare · doi:10.3390/healthcare10122397 · Cited by 1
“According to the idea of curriculum learning [28,31], BUDD, BSDD and BIDD will be learned sequentially.”
MethodReview-Based Curriculum Learning for Neural Machine Translation
Hui, Z., Feng, C., & Zhang, T. (2022) · Communications in computer and information science · doi:10.1007/978-981-19-7960-6_3 · Cited by 1
New Objects on the Road? No Problem, We'll Learn Them Too
Singh, D. K., Rai, S. N., Joseph, K. J., et al. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981886
“Curriculum Learning [1], [8] is a training method that progressively trains a model from easy to hard samples.”
BackgroundCluster-based Curriculum Learning for Vision-and-Language Navigation
Wang, T., Wu, Z., Liu, Z., & Wang, D. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892100 · Cited by 1
“Inspired by human learning, the training paradigm of CL is proposed [12].”
BackgroundUniversal Graph based Distantly Supervised Relation Extraction
Dai, Q., Heinzerling, B., Inoue, N., & Inui, K. (2022) · Journal of Natural Language Processing · doi:10.5715/jnlp.29.1138
A Search-Enhanced Path Mining and Ranking Method for Cross-lingual Knowledge Base Question Answering
Wu, Z., Zhu, M., Zhang, M., Peng, S., Zhang, W., Zhu, T., Zhu, J., Li, P., Yang, H., & Qin, Y. (2022) · Communications in computer and information science · doi:10.1007/978-981-19-8300-9_4
Curriculum Learning in Peristaltic Sortation Machine
Hameed, M. S. A., Koneru, V. H., Pöppelbaum, J., & Schwung, A. (2022) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin51773.2022.9976094
“The idea being that the simpler tasks provide a global picture of the optimization problem, while difficult tasks reveal the actual highly complex non-convex optimization problem [11].”
BackgroundLearning to Aggregate and Refine Noisy Labels for Visual Sentiment Analysis
Zhu, W., Zheng, Z., Zheng, H., Lyu, H., & Luo, J. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956328
“Curriculum learning is also a possible method for robust learning [17], [18].”
MethodSample-Efficient Reinforcement Learning for Pose Regulation of a Mobile Robot
Brescia, W., Cicco, L. D., & Mascolo, S. (2022) · International Conference on Control, Automation and Information Sciences (ICCAIS) · doi:10.1109/iccais56082.2022.9990480
“An interesting approach recently proposed in the literature to tackle such an issue is Curriculum Learning (CL) [4].”
BackgroundCurriculum Contrastive Learning for COVID-19 FAQ Retrieval
Zhang, L., & Liu, J. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995534
“It has been shown that the scheduling strategy of learning tasks of different difficulty levels directly affects the performance of learners [30] [31].”
BackgroundMethodAccelerating Multi-Exit BERT Inference via Curriculum Learning and Knowledge Distillation
Gu, S., Luo, X., Wang, X., & Guo, Y. (2023) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194023500018
Improving Speech Separation with Knowledge Distilled from Self-supervised Pre-trained Models
Qu, B., Li, C., Bai, J., & Qian, Y. (2022) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp57327.2022.10038203
“We also use the curriculum learning [32] method to make better use of the predicted SSL features.”
MethodXu, R. (2022) · Digital Repository Service (Northeastern University) · doi:10.17760/d20467228
Heterogeneous Multi-unit Control with Curriculum Learning for Multi-agent Reinforcement Learning
Chen, J., Jiang, K., Liang, R., Wang, J., Zheng, S., & Tan, Y. (2022) · Communications in computer and information science · doi:10.1007/978-981-19-9297-1_1
Enhanced Reinforcement Learning Method Based on AlphaGo-Zero
Jin, X., & Liu, W. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-0479-2_10 · Cited by 1
DCD: A New Framework for Distillation Learning With Dynamic Curriculum
Li, J., Zhang, Y., Li, Y., Gong, X., & Wang, W. (2022) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc-dss-smartcity-dependsys57074.2022.00149
“Original CL [27] proposed a predefined complexity indicator and presented low complex examples first and gradually introduced more complex samples in training process.”
BackgroundWang, J., & Chen, Y. (2022) · Machine learning · doi:10.1007/978-981-19-7584-4_14
Deep curriculum learning for follow-up MRI registration in glioblastoma
Banerjee, S., Toumpanakis, D., Dhara, A. K., et al. (2023) · SPIE Medical Imaging: Image Processing · doi:10.1117/12.2654143
“Bengio et al. 9 proposed CL as an approach for optimizing machine learning models.”
MethodMachine Learning Curriculums Generated by Classifier Ensembles
Huang, T.-J., & Kameyama, K. (2023) · IEEE International Colloquium on Signal Processing & Its Applications (CSPA) · doi:10.1109/cspa57446.2023.10087822
“The same progress is applied in improving machine learning models, which has been found advantageous to assist supervised learning [4].”
BackgroundAn Alternative Curriculum Learning Approach with Macro Actions for Deep Reinforcement Learning
Song, J., & Ji, J. (2022) · doi:10.1109/cecit58139.2022.00016
“On the other hand, curriculum learning [9] is also considered as a useful tool for DRL to accelerate the learning process for a sparse-reward task.”
MethodActive Learning Strategies Based on Text Informativeness
Li, R., Yamakata, Y., & Tajima, K. (2022) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat55865.2022.00015
“[15] proposed the concept of curriculum learning, which also manages the training order in supervised machine learning.”
BackgroundSachetti, N., Lima, B. M., Berndt, A., et al. (2023) · Anais do Computer on the Beach · doi:10.14210/cotb.v14.p456-458
An Asynchronous Updating Reinforcement Learning Framework for Task-Oriented Dialog System
Zhang, S., Hu, Y., Wang, X., & Yuan, C. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096940
“In addition, we use curriculum learning [8] to boost the DST learning by gradually increasing the complexity of the data samples used during the training process.”
MethodA Pipelined Framework with Serialized Output Training for Overlapping Speech Recognition
Li, T., Huang, L., Wang, F., Li, S., Hong, Q., & Li, L. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-2401-1_10
Withdrawals Prediction in Virtual Learning Environments with Deep Self-paced Learning
Li, X., Xu, W. H., & Xie, S. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-2449-3_7
Tan, X. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-981-99-0827-1_3
Human-Machine Co-Learning : Case Study on Motor Skill Acquisition
Sungeelee, V., Loriette, A., Sigaud, O., et al. (2023) · doi:10.1145/3583961.3583977
“Cette technique a été utilisée dans le cadre de l’apprentissage machine supervisé [4, 17] ainsi que l’apprentissage par renforcement [14], mais peu d’utilisations existent dans le cas d’interaction avec des utilisateurs apprenants.”
MethodCascaded Perceptual Networks for Low Dose CT Denoising
Ataei, S. (2023) · doi:10.32920/23332967
Learning to Correct Erroneous Words for Document Grounded Conversations
Qiu, J., Zhang, H., & Yang, Y. (2023) · International Conference on Software and Computer Applications (ICSCA) · doi:10.1145/3587828.3587883
“Inspired by human education that starting from easy and gradually learning more complex concepts, we incorporate curriculum learning [2] into the response generation process for better generalization capacities.”
Witte, S. D., Lefebvre, T., Hauwermeiren, T. V., & Crevecoeur, G. (2023) · IEEE/ASME International Conference on Advanced Intelligent Mechatronics · doi:10.1109/aim46323.2023.10196273
“The latter is harder to learn, therefore a simplified form of curriculum learning [17] is implemented.”
BackgroundCombining Transformers with Natural Language ExplanationsPreprint
Ruggeri, F., Lippi, M., & Torroni, P. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4531749 · Cited by 3
Hybrid-Regressive Paradigm for Accurate and Speed-Robust Neural Machine Translation
Wang, Q., Hu, X., & Chen, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.367 · Cited by 1
“Curriculum Learning To ensure the model is not overly biased towards auxiliary tasks, we propose gradually transferring the training tasks from auxiliary tasks to primary tasks through curriculum learning (Bengio et al., 2009).”
MethodGarg, S., Sasaoka, S., Dominguez, D. S., et al. (2023) · International Cosmic Ray Conference · doi:10.22323/1.444.1536
“[2] in their, “Discussing merits and demerits of various training methodologies for Artificial Neural Networks".”
BackgroundMulti-Task Curriculum Learning for Partially Labeled Data
Jang, W.-D., Lukyanenko, S., Wei, D., Yang, J., Leahy, B., Yang, H. Y., Ben‐Yosef, D., Needleman, D. J., & Pfister, H. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230365
“Curriculum learning [18] optimizes the learning order of data in training.”
Alignment Offset Based Adaptive Training for Simultaneous Machine Translation
Liang, Q., Liu, Y., Meng, F., Xu, J., Chen, Y., & Zhou, J. (2023) · doi:10.1109/icnlp58431.2023.00035
“Reference [14] proposes a machine learning training strategy, named curriculum learning (CL).”
BackgroundMethodLabel-noise learning via mixture proportion estimation
ZHENG, Q., Cao, S., Ruan, J., Zhao, R., & Dong, B. (2023) · Scientia Sinica Informationis · doi:10.1360/ssi-2023-0126
Fast Adversarial Training with Dynamic Batch-level Attack Control
Jung, J., Song, J., Jang, H., Lee, H., Choi, K., Park, N., & Lee, J. (2023) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac56929.2023.10247930
“This is similar to i) curriculum learning [18] where the model starts learning from easy samples and gradually moves on to more difficult samples, or ii) the popular learning rate warm-up [19] scheme.”
ResultTowards English-centric Zero-shot Neural Machine Translation: The Analysis and Solution
Wang, S. (2023) · IEEE International Conference on Sensors, Electronics and Computer Engineering (ICSECE) · doi:10.1109/icsece58870.2023.10263323
Collaborative 3D Object Detection
Chen, S., & Hu, Y. (2023) · Advances in computer vision and pattern recognition · doi:10.1007/978-981-99-4287-9_6
Open-Vocabulary Object Detection Via Debiased Curriculum Self-TrainingPreprint
Zhang, H., Guan, D., Ke, X., et al. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4612990
TabMentor: Detect Errors on Tabular Data with Noisy Labels
Zhang, Y., Qin, J., Wang, Y., Ali, M. A., Ji, Y., & Mao, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46671-7_12
Gradual Syntactic Label Replacement for Language Model Pre-Training
Wang, Y., Zhang, Y., Li, P., & Liu, Y. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3331096 · Cited by 2
“Inspired by psycho-linguistic curriculum learning [15], [16], we propose two main strategies that use different training corpus at stages: 1) common words first, rare words next, and 2) additional syntactic knowledge is used to help the learning process.”
BackgroundMethodImproving ResNet Model Accuracy with Curriculum Learning Using Blurred Images
Yin, Z., Zeng, Q., Pan, Q., & Zhuang, Y. (2023) · International Conference on High Performance Compilation, Computing and Communications (HP3C) · doi:10.1145/3606043.3606086
“It involves presenting training examples arranged in increasing difficulty, and has been shown to improve the efficiency and effectiveness of model training[2] [8].”
BackgroundCurriculum Learning Based Approach for Faster Convergence of TTS Model
Kaur, N., & Ghosh, P. K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-48312-7_17
All You See Is the Tip of the Iceberg: Distilling Latent Interactions Can Help You Find Treasures
Cai, Z., Yuan, G., Zhuang, X., et al. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-8181-6_19
On Skull-Closed Machine Thinking Based on Emergent Turing Machines
Wu, X., Zheng, Z., & Weng, J. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3337322 · Cited by 1
“This learning strategy is similar with the curriculum learning (CL), especially the CL in RL domain [33], [34].”
ResultData Selection Curriculum for Abstractive Text Summarization
Sun, S., Yuan, R., He, J., Cao, Z., Li, W., & Jia, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.537
“This study aims to explore the potential for optimizing ATS models through the strategic utilization of data selection and curriculum learning (Bengio et al., 2009) in conjunction with contrastive learning techniques.”
Oba, M., Haga, A., Fukatsu, A., & Oseki, Y. (2023) · BabyLM Challenge · doi:10.18653/v1/2023.conll-babylm.25 · Cited by 6
Feasibility Evaluation of Oversize Load Transportation Using Conditional Rewarded Deep Q-Networks
Son, H., Kim, J., Jung, H., et al. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3339143
“To make the model converge fast, curriculum learning is conducted [37].”
MethodLearning approaches and tricks
Ros, F., & Riad, R. (2023) · Unsupervised and semi-supervised learning · doi:10.1007/978-3-031-48743-9_7
Curriculum Learning with Class-Label Composition for Weakly Supervised Semantic SegmentationPreprint
Hwang, D., Kim, H., Baek, D., Kim, H., Kye, I., & Choe, J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4671726
Evaluating Optimization Approaches for Deep-Reinforcement-Learning-based Navigation Agents
Kästner, L., Roberts, L., Bhuiyan, T., & Lambrecht, J. (2023) · IEEE Conference on Cybernetics and Intelligent Systems · doi:10.1109/cis-ram55796.2023.10370762 · Cited by 1
“Another important approach to accelerate DRL training is curriculum learning proposed by Bengio et al. [15].”
BackgroundSeagrass Classification Using Unsupervised Curriculum Learning (Ucl)Preprint
Abid, N., Noman, M. K., Kavacs, G., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4684221
Smart Library: A Multi-Agent Path Planning Logistics Operation using Reinforcement LearningPreprint
Batista, H., de Carvalho, K. B., de Oliveira, I. R. L., & Brandao, A. S. (2024) · Research Square · doi:10.21203/rs.3.rs-3854774/v1
Li, J.-H., Xiao, L., Luo, J., Liu, X., & Chen, J. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385376 · Cited by 2
Ma, Y., & Klabjan, D. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386172
“Auto-encoder has a long history [26] of pre-training artificial neural networks and is widely used in recent models [27].”
MethodEnvironment Adversarial Reinforcement Learning
Cooper, J. R. (2024) · AIAA SciTech Forum · doi:10.2514/6.2024-2405
Unsupervised Domain Adaptation on Sentence Matching Through Self-Supervision
Bai, G., Liu, Q., He, S., Liu, K., & Zhao, J. (2023) · Journal of Computer Science and Technology · doi:10.1007/s11390-022-1479-0
A Velocity Controller for Quadrotors Based on Reinforcement Learning
Hu, Y., Luo, J., Dong, Z., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-981-99-9119-8_37
Learning Cost-Adjusted Predictive Models with Margin-Based Framework
Anwar, N., Battula, M., Kaur, J., et al. (2024) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc60891.2024.10427738
AutoPooling: Automated Pooling Search for Multi-valued Features in Recommendations
Wei, H., Yang, Y., Ma, S., et al. (2024) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3616855.3635808
“DLRMs map these features to dense vectors named embeddings via multi-hot encoding and embedding-lookup [5, 6].”
BackgroundAutomated Curriculum Reinforcement Learning in Path-finding Scenarios
Chen, S., Cai, Y., Tong, E., Tian, Y., Li, Y., Niu, W., & Liu, J. (2023) · IEEE Smart World Congress (SWC) · doi:10.1109/swc57546.2023.10449185
“Strategies similar to curriculum learning have been applied in robotics [21], [22] and accelerated training for multi-agent games [23], [24].”
BackgroundChoi, T., Would, O., Salazar-Gomez, A., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4756089
Correct Like Humans: Progressive Learning Framework for Chinese Text Error CorrectionPreprint
Li, Y., Ma, S., Chen, S., Huang, H., Huang, S., Li, Y., Zheng, H.-T., & Shen, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4765255
Improving Iteration-based Non-Autoregressive Language Model With Time Step Awareness
Jiang, H., Cai, G., & Tang, X. (2023) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads60453.2023.00385
“Inspired by curriculum learning [16], we attempt to further improvement by a progressive learning strategy.”
MethodIskandar, A., & Kovács, B. (2024) · INTELIGENCIA ARTIFICIAL · doi:10.4114/intartif.vol27iss73pp163-176 · Cited by 2
“In the field of machine learning, the concept of curriculum learning (CL) was introduced by [6], highlighting the importance of organizing training examples in an orderly manner, from simpler to more complex tasks.”
BackgroundDomain Adaptation Curriculum Learning for Scene Text Detection in Inclement Weather Conditions
Liu, Y., Zhou, G., Tian, J., Deng, E., Meng, L., & Jia, Z. (2024) · IEEJ Transactions on Electrical and Electronic Engineering · doi:10.1002/tee.24036
“Bengio et al. [8] applied this approach to artificial intelligence, mimicking the easy-to-hard learning strategies used by humans.”
BackgroundLightweight Delivery Detection on Doorbell Cameras
Khorramshahi, P., Wu, Z., Wang, T., et al. (2024) · IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv57701.2024.00681
“Inspired by the works of [7, 20, 35] which incorporate the paradigm of curriculum learning [3] for object detection and vehicle analytics tasks, we devise a training mechanism to simplify the learning of delivery versus non-delivery events at early stages of optimization and gradually make the task…”
MethodCho, W.-W., Choi, D., Lim, H., Choi, J., Choi, S., Min, H., Lim, S., & Choo, J. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00744 · Cited by 1
ProcSim: Proxy-based Confidence for Robust Similarity Learning
Barbany, O., Lin, X., Baştan, M., & Dhua, A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00134 · Cited by 2
“In the curriculum learning training, the samples are fed in increasing order of difficulty, which improves the speed of convergence and the quality of the models obtained [3,14].”
Discrete-time simulated annealing: A convergence analysis via the Eyring–Kramers law
Tang, W., Wu, Y., & Zhou, X. Y. (2024) · Numerical Algebra Control and Optimization · doi:10.3934/naco.2024015
The limitations of automatically generated curricula for continual learning
Kravchenko, A., & Cusack, R. (2024) · PLoS ONE · doi:10.1371/journal.pone.0290706
Weighted Self-Paced Learning with Belief FunctionsPreprint
Zhang, S., Han, D., Dezert, J., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4798521
Feedback-Based Curriculum Learning for Collision Avoidance
Choi, J., Hwang, G., & Eoh, G. (2024) · IEEE Access · doi:10.1109/access.2024.3391408 · Cited by 1
DTN-MTLF: A dual-teacher network based multi-task learning framework for unbiased glaucoma diagnosis
Liu, Z., Guo, F., Yang, J., Ge, X., & Tang, J. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19218-4
SEE: Spherical Embedding Expansion for Improving Deep Metric Learning
Le, B. M., & Woo, S. S. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-2253-2_11 · Cited by 2
Präntare, F. (2024) · Linköping studies in science and technology. Dissertations · doi:10.3384/9789180756013
Data Quality-based Gradient Optimization for Recurrent Neural Networks
Huang, F., Yi, P., Li, S., & Xu, H. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651918
“Bengio mentioned in his original work that the basic idea of such methods is to first train the model with easy samples, and then gradually increase the difficult samples until the whole training datasets [27].”
BackgroundMadduri, M. M., Yamagami, M., Li, S., Burckhardt, S. N., Burden, S. A., & Orsborn, A. L. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.05.23.595598
Wang, Y., Dang, K., Yang, R., et al. (2024) · Remote Sensing · doi:10.3390/rs16111961
Fedspl: Robust Federated Learning Against Noisy Labels Via Self-Paced LearningPreprint
Chen, Y., Da, Z., Jiang, X., Zhang, Y., & Xiong, N. N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4853275
Cl-Notears:Continuous Optimization Algorithm Based on Curriculum Learning FrameworkPreprint
Liu, K., Liu, L., Xiao, K., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4855395
Dstf: Enhancing Traffic Flow Prediction with a Diversified Spatio-Temporal Feature Extraction ModelPreprint
Wang, X., Wang, X., Zou, F., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4867831
Scaling Up Multi-Agent Reinforcement Learning via Graph Decomposition Invariant Network
Fu, H., & Ji, J. (2024) · International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) · doi:10.1109/ainit61980.2024.10581435
“One approach to tackle the training challenges of large-scale multi-agent systems is curriculum learning [4].”
BackgroundDual-Resolution Feature Fusion Based on Curriculum Learning
Li, J., Li, F., Liu, C., & Li, X. (2024) · International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) · doi:10.1109/ainit61980.2024.10581752
“The strategy of curriculum learning, first introduced by Bengio et al [8], aims to organize input data or tasks in an orderly manner, following a sequence from simple to complex, to enhance model learning efficiency.”
MethodGradual Batch Alternation for Effective Domain Adaptation in LiDAR-Based 3D Object Detection
Rochan, M., Chen, X., Grandhi, A., et al. (2024) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55156.2024.10588523
“Lastly, our work is related to curriculum learning [15] and a recent research in natural language processing [16] that highlight the role of ordering in the training data for model learning.”
BackgroundAccumulating Curriculum Knowledge Based on Neural Network Isolated Algorithm
Liou, C.-F., & Yu, Y. (2024) · IEEE International Symposium on Industrial Electronics (ISIE) · doi:10.1109/isie54533.2024.10595804
“In the early work, Bengio et al. [5] ranked difficulty with the number of irrelevant nonzero inputs or the value of margin.”
BackgroundFedspl: Robust Federated Learning Against Noisy Labels Via Self-Paced LearningPreprint
Chen, Y., Da, Z., Jiang, X., Zhang, Y., & Xiong, N. N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4904916
Boosting Multitask Decomposition: Directness, Sequentiality, Subsampling, Cross-Gradients
Millinghoffer, A., Antal, M., Marosi, M., Formanek, A., Antos, A., & Antal, P. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-66538-7_3 · Cited by 1
Tao, L., Kang, M., Dong, J., Zhang, S., Ye, K., Chen, S., & Zheng, N. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610745 · Cited by 1
“CM3 adopts two-stage course learning [18] [19] and a specially designed counterfactual baseline.”
Background
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.