Curriculum learning
The DOI matches its Crossref record; 35 of the 40 references it cites were found in a registry or an index.
Abstract
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moreover, the lack of acoustical signals could even account for spatial disorientation 91,92.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [92]
In the citing paper
“This suggests that curriculum-style learning (Bengio et al., 2009), where simpler or archetypal problems are presented first to build a repository of valid heuristics, may potentially bootstrap performance.”
From Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2025) · cites this paper as Bengio et al., 2009
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“In Collobert and Weston (2008), the main objective is to learn an embedding for the words as a side-e.ect of learning to compute this score.”
BackgroundMethodInfluentialSelf-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationLearning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 8×
“The idea of training a learning machine with a curriculum can be traced back at least to Elman (1993).”
BackgroundMethodInfluentialRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Research works printed but not linked: 3
- 10.5555/89603 · doi:10.5555/89603
- 10.5555/866740 · doi:10.5555/866740
- 10.5555/525571 · doi:10.5555/525571
Cited by
Items 4,301–4,400 of 5,240
TSO: Curriculum Generation using continuous optimization.Preprint
Sarkar, D., & Gupta, M. (2021) · arXiv (Cornell University)
Automatic Curricula via Expert DemonstrationsPreprint
Dai, S., Hofmann, A., & Williams, B. C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09159 · Cited by 12
“Curriculum learning [8] is a continual learning method that accelerates the learning progress by gradually increasing the task difficulty.”
BackgroundSolving Graph-based Public Good Games with Tree Search and Imitation LearningPreprint
Darvariu, V.-A., Hailes, S. M. V., & Musolesi, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06762 · Cited by 11
“We employ curriculum learning, a methodology successful in a variety of machine learning settings [7, 53], and compare against training only on the target size as well as mixing the examples of different sizes instead of constructing a curriculum.”
MethodDual Graph enhanced Embedding Neural Network for CTR PredictionPreprint
Guo, W., Su, R., Tan, R., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.00314
“To solve these issues, we design an organized learning mechanism by taking inspiration from the curriculum learning, which introduces different concepts at different time and then uses the previous learned concepts to promote the learning of new concepts [1].”
Hi-Phy: A Benchmark for Hierarchical Physical ReasoningPreprint
Xue, C. Q., Pinto, V., Gamage, C., Zhang, P., & Renz, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09692 · Cited by 1
“Humans learn much better when examples are presented in a meaningful order with increasing 85 complexity than when examples are presented randomly [25].”
BackgroundWeighting Adversarial Neural Network for Domain Adaptation in RegressionPreprint
de Mathelin, A., Richard, G., Deheeger, F., et al. (2021) · HAL (Le Centre pour la Communication Scientifique Directe)
Explorations In Curriculum Learning Methods For Training Language Models
Campos, D. (2020) · ResearchWorks at the University of Washington (University of Washington)
Progressive Class-Based Expansion Learning for Image Classification
Wang, H., Zhao, H., & Li, X. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3094174 · Cited by 2
“To date, curriculum learning [10]–[15] and self-paced learning [16]–[22] have been proposed to improve the speed of convergence of the training process to a minimum and the quality of the local minima obtained.”
BackgroundIs Active Learning Always Beneficial? (Student Abstract)
Kravchenko, A., & Cusack, R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i18.17906
“In machine learning, this is called automated curriculum learning (Bengio et al. 2009); in human learning and developmental robotics it is active or curiosity-driven learning.”
BackgroundCross-domain error minimization for unsupervised domain adaptationPreprint
Du, Y., Chen, Y., Cui, F., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.15057
“Moreover, we propose to use curriculum learning [2] based strategy to select target samples with high prediction confidence during training.”
MethodCurriculum-Driven Multi-Agent Learning and the Role of Implicit Communication in Teamwork.Preprint
Grupen, N. A., Lee, D. D., & Selman, B. (2021) · arXiv (Cornell University) · Cited by 1
Learning Algebraic Recombination for Compositional GeneralizationPreprint
Liu, C., An, S., Lin, Z., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.06516
“To help the model converge better, we use a simple curriculum learning (Bengio et al., 2009) strategy to train the model.”
MethodMehrasa, N. (2021) · Summit (Simon Fraser University)
Exploring Dropout Discriminator for Domain AdaptationPreprint
Kurmi, V. K., Subramanian, V. K., & Namboodiri, V. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.04231
“The curriculum learning [45] enhances model’s performance and its generalization capability.”
BackgroundMethodLiew, A., Lee, C. C., Lan, B. L., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.04099
“Curriculum learning advocates learning tasks with increasing difficulty to improve generalization of the network [40].”
BackgroundDeep Reinforcement Learning for Task Planning of Virtual Characters
Souza, C., & Velhor, L. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-80126-7_49
“Yet, this 12Curriculum learning is an approach of increasing the environment difficulty according with the agent learning which was proposed in (Bengio et al., 2009) exposition need to be done without a steep difficulty variation, or the agent may not be able to learn anything.”
BackgroundA Comprehensive Study on Novel Video Frame Interpolation Methods
Mahajan, H., Shekhadar, Y., Silvister, S., Komandur, D., & Pise, N. N. (2021) · International Journal of Computer Applications · doi:10.5120/ijca2021921472
“The curriculum learning method [8] has been used while training the network model.”
MethodLearning to Transfer: A Foliated TheoryPreprint
Petangoda, J. C., Deisenroth, M. P., & Monk, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.10763
“Curriculum learning (Bengio et al., 2009) is an interesting iteration on this sequential learning problem.”
BackgroundCompensation LearningPreprint
Yao, R., & Wu, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.11921
“Weighting is a widely used machine learning strategy in at least the following five areas, including, noisy-aware learning [29], curriculum learning [1], crowdsourcing learning [5], cost-sensitive learning [3], and imbalance learning [18].”
Multi-stage Pre-training over Simplified Multimodal Pre-training ModelsPreprint
Liu, T., Feng, F., & Wang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.14596 · Cited by 3
“Existing research on curriculum learning (Bengio et al., 2009) has shown that imitating the process of human learning by gradually increasing the difficulty of a task from simple to complex in stages helps to make better use of different types of data and effectively improve the performance of learning.”
Learning to Design and Construct Bridge without BlueprintPreprint
Li, Y., Kong, T., Li, L., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.02439
“Curriculum learning [22] is a common practice in many robotic learning tasks [23], [24] by always training the agent with tasks of moderate difficulty.”
BackgroundUniform Sampling over Episode DifficultyPreprint
Arnold, S. M. R., Dhillon, G. S., Ravichandran, A., & Soatto, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.01662 · Cited by 9
“They use this mechanism to build a curriculum [2, 63] of increasingly difficult episodes.”
BackgroundSpatial Transformer Networks for Curriculum Learning
Nies, J.-F., Azimi, F., Palacio, S., Raue, F., Hees, J., & Dengel, A. R. (2022) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta56598.2022.10034595
“investigated the effect of starting from easier examples in the task of language acquisition [23] and in [4], the authors studied where and when curriculum learning can be beneficial for the training process in the context of shape recognition tasks.”
BackgroundMethodContrastive Representations for Label Noise Require Fine-TuningPreprint
Nodet, P., Lemaire, V., Bondu, A., & Cornuéjols, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.09154
“An example of SSL algorithm is Contrastive Learning [21], where a representation of the data is learned by making feature vectors from similar pictures (i.e. generated from the same original picture by using two different transformer functions) to be close in the feature space whereas feature vectors from dissimilar p…”
MethodInfluentialProgressive Learning With Anchoring Regularization For Vehicle Re-Identification
Besbes, M. D. E., Tabia, H., Kessentini, Y., et al. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506152
“Bengio et al. [7] proposed to arrange the learning process from easier to more complex tasks, so the learning should be organised in a meaningful order which gradually illustrates more concepts.”
BackgroundMethodInfluentialSentiment Preservation in Review Translation using Curriculum-based Re-inforcement Framework
Kumari, D., Chennabasavaraj, S., Garera, N., & Ekbal, A. (2021)
Matsunami, N., Okuhara, S., & Itō, T. (2021) · Transactions of the Japanese Society for Artificial Intelligence · doi:10.1527/tjsai.36-5_e-k62
Learning Curricula for Multilingual Neural Machine Translation Training.
Kumar, G., Koehn, P., & Khudanpur, S. P. (2021) · Cited by 1
Applying Soft Attention in Deep Reinforcement Learning for Robot Collision Avoidance
Xu, P., Chen, G., & Ji, J. (2021) · International Conference on Mathematics and Artificial Intelligence (ICMAI) · doi:10.1145/3460569.3460573
“Curriculum learning [16] is a learning strategy in machine learning, which advocates the model should start from easy samples and gradually advance to the complex instances.”
BackgroundIndividual and Collective Autonomous DevelopmentPreprint
Lippi, M., Mariani, S., Martinelli, M., & Zambonelli, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.11223
DCUR: Data Curriculum for Teaching via Samples with Reinforcement LearningPreprint
Seita, D., Gopal, A., Mandi, Z., & Canny, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.07380 · Cited by 1
“[5] confirmed these results by accelerating classification and language modeling by arranging samples in order of difficulty.”
ResultLearning to Aggregate and Refine Noisy Labels for Visual Sentiment AnalysisPreprint
Zhu, W., Zheng, Z., Zheng, H., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.07509
“Curriculum learning is also a possible method for robust learning [17], [18].”
MethodYES SIR!Optimizing Semantic Space of Negatives with Self-Involvement RankerPreprint
Pu, R., Zhang, X., Lai, R., Guo, Z., Zhang, Y., Jiang, H., Wu, Y., Yantao, J., Dou, Z., & Cao, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.06436 · Cited by 3
“…2021) showed that a proper selection of negative samples is essential to training a ranking model by fine-tuning PLMs. Motivated by human learning process and curriculum learning (Chevalier-Boisvert et al. 2018; Bengio et al. 2009), we observe that human tend to learn knowledge from easy to hard.”
BackgroundSemi-Supervised Text Classification via Self-PretrainingPreprint
Karisani, P., & Karisani, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.15300 · Cited by 3
Weakly Supervised Keypoint DiscoveryPreprint
Ryou, S., & Perona, P. (2021) · CaltechAUTHORS (California Institute of Technology) · doi:10.48550/arxiv.2109.13423 · Cited by 7
Inaguma, H., Dalmia, S., Yan, B., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.12804
“Therefore, we reason that CER thresholding is a type of curriculum learning [68] because CTC predictions are used gradually as training progresses.”
BackgroundMASAI: Multi-agent Summative Assessment Improvement for Unsupervised Environment Design
Wang, Y., & Haworth, M. B. (2021) · International Conference on Machine Learning
TSK Fuzzy System Towards Few Labeled Incomplete Multi-View Data ClassificationPreprint
Zhang, W., Deng, Z., Lou, Q., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05610
“7) MMCL: This algorithm integrates the label propagation learning and course learning [46], and takes the pseudo label of each round as feedback to guide the model learning.”
BackgroundWhich Samples Should be Learned First: Easy or Hard?Preprint
Zhou, X., & Wu, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05481
“CL [25], [29] is motivated by human learning that easy samples should be learned first.”
BackgroundOn Hard Episodes in Meta-LearningPreprint
Basu, S., Sharaf, A., Fusi, N., & Feizi, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.11190 · Cited by 1
“Curriculum learning (Hacohen & Weinshall, 2019; Bengio et al., 2009) aims to mimic the learning process of humans and animals.”
Estimating Similarity between Visual and Long Wave Infrared patches using Siamese CNN
Jyothi, C. S., & Sandhya, B. (2021) · International Conference on Emerging Techniques in Computational Intelligence (ICETCI) · doi:10.1109/icetci51973.2021.9574058 · Cited by 1
Confidence-Aware Active Feedback for Efficient Instance SearchPreprint
Zhang, Y., Liang, C. C., & Jiang, L. (2021) · arXiv (Cornell University)
Enhancing Reinforcement Learning with discrete interfaces to learn the Dyck LanguagePreprint
Dietz, F., & Klakow, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14350
“Learning in such a manner is much closer to the way humans learn, and has been shown to speed up learning and improve robustness(Bengio et al., 2009).”
BackgroundProgressive Transformer-Based Generation of Radiology Reports
Nooralahzadeh, F., Gonzalez, N. A. P., Frauenfelder, T., Fujimoto, K., & Krauthammer, M. O. (2021) · Zurich Open Repository and Archive (University of Zurich) · doi:10.48550/arxiv.2102.09777 · Cited by 61
“, image-to-text-to-text) generation framework (inspired by Curriculum Learning (Bengio et al., 2009) and the work of Tan et al.”
BackgroundMethodCompetence-based Curriculum Learning for Multilingual Machine TranslationPreprint
Zhang, M., Meng, F., Tong, Y., & Zhou, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.04002 · Cited by 3
“Curriculum learning was first proposed by Bengio et al. (2009) with the idea of learning samples from easy to hard to get a better optimized model.”
BackgroundGeneralization in Text-based Games via Hierarchical Reinforcement LearningPreprint
Xu, Y., Fang, M., Chen, L., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.09968
“The scheduled task sampling is inspired by the curriculum learning (Bengio et al., 2009), where we schedule the tasks based on their difficulties.”
BackgroundToken-wise Curriculum Learning for Neural Machine TranslationPreprint
Chen, L., Jiang, H., Liu, X., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11088
“Aside from previous approaches, curriculum learning (Bengio et al., 2009) is proposed to address the data insufficiency issue via utilizing the limited data more efficiently (Zhang et al., 2019b).”
MethodExploiting Curriculum Learning in Unsupervised Neural Machine TranslationPreprint
Lu, J., & Zhang, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.11177
“Curriculum learning (Bengio et al., 2009) is motivated by the learning strategy of biological organisms which orders the training samples in an easy-to-hard manner (Elman, 1993).”
BackgroundDiverse Distributions of Self-Supervised Tasks for Meta-Learning in NLPPreprint
Bansal, T., Gunasekaran, K. P., Wang, T., Munkhdalai, T., & McCallum, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.01322 · Cited by 32
“(Bengio et al., 2009; Graves et al., 2017) instead posits that choosing the order in which instances are presented, with gradually increasing complexity, can enable faster learning and better generalization.”
Improving Sequence-to-Sequence Pre-training via Sequence Span RewritingPreprint
Zhou, W., Ge, T., Xu, C., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.00416
“the SSR training examples with curriculum learning (Bengio et al., 2009) according to their difficulties.”
BackgroundMethodBandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed BanditsPreprint
Kreutzer, J., Vilar, D., & Sokolov, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.06997
“Such approaches are brittle in that they may not generalize well across tasks, and there has been evidence that even the reverse of the initially hypothesized order works well (Bengio et al., 2009; Wang et al., 2018; Zhang et al., 2018).”
Automatic Goal Generation using Dynamical Distance LearningPreprint
Prakash, B., Waytowich, N. R., Mohsenin, T., & Oates, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.04120 · Cited by 2
“Bengio et al. (2009) show that it can help with both speed of convergence as well as accuracy of the final solution. In the reinforcement learning domain Matiisen et al. (2019) propose a Teacher-Student framework for automatic curriculum learning where the teacher helps the student to learn a complex task by proposing…”
BackgroundMethodInfluentialAccelerating Deep Learning with Dynamic Data PruningPreprint
Raju, R., Daruwalla, K., & Lipasti, M. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12621 · Cited by 72
“Furthermore, curriculum learning [2] asserts that samples can be ranked which might allow us to prune redundant “easy” samples.”
BackgroundSemi-Supervised Learning with Taxonomic LabelsPreprint
Su, J.-C., & Maji, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.11595
“Such labels can also be added gradually as a form of curriculum learning [2, 6] to reduce model drift.”
BackgroundTraining Efficiency and Robustness in Deep LearningPreprint
Faghri, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01423 · Cited by 4
How Curriculum Learning Performs on AMR Parsing
Bui, M.-Q. V., Tran, V., Nguyen, H.-T., Dang, T.-B., & Nguyen, L.-M. (2021) · International Conference on Knowledge and Systems Engineering/Proceedings · doi:10.1109/kse53942.2021.9648646
Modeling nuisance classifier towards class-incremental learning of crowd-sourced data
Tabib, R. A., Santoshkumar, T., Hegde, D., Jamadandi, A., & Mudenagudi, U. (2021) · Indian Conference on Computer Vision, Graphics and Image Processing · doi:10.1145/3490035.3490292
“The Curriculum learning model [1] is inspired by the biological process in which humans and animals are presented with learning data which is gradually increased in the number of samples as well as increasing complexity.”
MethodGym Hero: A Research Environment for Reinforcement Learning Agents in Rhythm Games
Filho, R. F. F., Nogueira, Y. L. B., Vidal, C. A., Cavalcante-Neto, J. B., & Serafim, P. B. S. (2021) · Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) · doi:10.1109/sbgames54170.2021.00020
Curriculum Learning-based Object Transportation using Region Partitioning
Eoh, G., & Park, T.-H. (2021) · International Conference on Control, Automation and Systems (ICCAS) · doi:10.23919/iccas52745.2021.9649829
Image Block Regression Based on Feature Fusion for CNN-Based Spatial Steganalysis
Chen, Z., Yu, X., & Chen, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-95398-0_18
Predictive Modeling for Complex Care Management
Mavroudeas, G., Neehal, N., Shou, X., Magdon‐Ismail, M., Kuruzovich, J., & Bennett, K. P. (2021) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm52615.2021.9669714 · Cited by 1
Dynamic Curriculum Learning with Co-training for Medical Dialogue Generation
Zhu, Q., Tan, Z., Duan, J., Wu, P., Zhao, D., & Liu, J. (2021) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm52615.2021.9669462 · Cited by 1
Stereo Image Super-Resolution Based on Pixel-Wise Knowledge Distillation Strategy
Ma, L., & Li, S. (2021) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip53242.2021.9675446
Discovering the arrow of time in machine learning
Kasmire, J., & AnranZhao (2021) · doi:10.21428/7a45813f.c10e4455
“, easy or straightforward training data first and then hard or more ambiguous training data later) [39].”
BackgroundCurriculum Self-supervised Learning for Weakly-supervised Histopathological Image Segmentation
Cheng, J., Mai, J., Pan, X., et al. (2021) · International Conference on Internet Computing for Science and Engineering · doi:10.1145/3485314.3485331
“Inspired by Bengioet al.[21], we proposed anovel strategy named CSSL(see Figure 1).”
MethodHuman-Agent Collaboration Strategies for Vision-Grounded Instruction Following
Chao, G.-L., & Lane, I. R. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688143
HiTNet: Byte-to-BPE Hierarchical Transcription Network for End-to-End Speech Recognition
Gowda, D., Garg, A., Kim, J., Kumar, M., Singh, S., Gupta, A., Kumar, A., Dawalatabad, N., Maghan, A., Singh, S., et al. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9687943
Future of Affective Computing and Applied Affective Computing
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502411 · Cited by 1
“These models are often inspired by concepts in biology, developmental psychology, and cognitive science, such as memory and forgetting [Parisi et al. 2017], curiosity [Ceha et al. 2019], curriculum learning [Bengio et al. 2009], and knowledge transfer [Pan and Yang 2009].”
BackgroundMultimodal Data Collection and Processing for Applied Affective Computing
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502403
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502399
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502412
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502413
Applied Affective Computing in Built Environments
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502409 · Cited by 1
Machine Learning Approaches for Applied Affective Computing
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502402
Affective Computing for Enhancing Well-Being
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502408 · Cited by 1
Synthesizing Natural and Believable Emotional Expressions
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502406
Addressing Ethical Issues of Affective Computing
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502410
Emotions as Studied in Psychology and Cognitive Science
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502401
(2021) · Wiley eBooks · doi:10.1002/9781119646181.biblio
Transfer Dynamics in Emergent Evolutionary Curricula
Dharna, A., Hoover, A. K., Togelius, J., & Soros, L. B. (2022) · IEEE Transactions on Games · doi:10.1109/tg.2022.3151025 · Cited by 9
“Transfer learning exploits this shared structure to increase performance or decrease training time on the target task [7]–[12].”
BackgroundDisentangled Representation Learning from Videos for Facial Action Unit Detection
Dong, Z., Zhao, H., Juan, J., He, S., & Ma, Z. T. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-16-6963-7_53
Adaptable Text Matching via Meta-Weight Regulator
Zhang, B., Zhang, C., Ma, F., & Song, D. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531932
“Weighting examples is also applied to curriculum learning and regularization of noisy and corrupted labels [3, 13, 30].”
BackgroundAdaptive Locomotion Control of Sixteen-legged Robot based on Deep Reinforcement Learning
Mu, X., Shao, S., & Zhang, D. (2021) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio54168.2021.9739526 · Cited by 2
“We use the curriculum learning technique [20] for training policy, which makes learning smooth, efficient, and effective.”
MethodLi, C., Liu, Y., Bing, Z., Seyler, J. R., & Eivazi, S. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-030-97672-9_42
Quantifying Appropriateness of Summarization Data for Curriculum Learning
Kano, R., Taniguchi, T., & Ohkuma, T. (2022) · Journal of Natural Language Processing · doi:10.5715/jnlp.29.144 · Cited by 1
Research on Chinese Text Error Correction Based on Transformer Enhanced Architecture
杨, 靖. (2022) · Computer Science and Application · doi:10.12677/csa.2022.123057
Effective Use of Target-side Context for Neural Machine Translation
Mino, H., Ito, H., Goto, I., et al. (2021) · Journal of Natural Language Processing · doi:10.5715/jnlp.28.1162
Lin, L., Zhang, D., Luo, P., et al. (2019) · Springer eBooks · doi:10.1007/978-981-13-2387-4_9
Improving De Novo Molecular Design with Curriculum LearningPreprint
Guo, J., Fialková, V., Arango, J. D., et al. (2021) · doi:10.33774/chemrxiv-2021-37m7w
Faculty Opinions recommendation of A developmental approach to machine learning?
Moore, D. (2020) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.732332264.793574321
Neural machine translation with limited resources
Mohiuddin, T. (2022) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/157475
Sethi, A., Gond, A. K., Singh, A. K., & Shambharkar, P. G. (2022) · International Conference on Applied Artificial Intelligence and Computing (ICAAIC) · doi:10.1109/icaaic53929.2022.9793209
“In very simple words, anti-curriculum training [20] involves training on ‘tough’ tasks before training on ‘easy’ ones.”
BackgroundMethodInfluentialPseudo Reward and Action Importance Classification for Sparse Reward Problem
Wu, Q., Feng, D., Zhai, Y., Ding, B., & Luo, J. (2022) · International Conference on Machine Learning and Computing (ICMLC) · doi:10.1145/3529836.3529918
“To relieve the sparse reward problem, many methods have been proposed and achieved good results in recent years, such as re-warding shaping[Ng 2003], curriculum learning[Bengio et al. 2009], hierarchical DRL[Kulkarni et al. 2016], etc.”
MethodExploring Cross-fusion and Curriculum Learning for Multi-modal Human Detection on Drones
Safa, A., Ocket, I., Catthoor, F., & Gielen, G. E. (2022) · doi:10.1145/3522784.3522785 · Cited by 1
Learning Advisor Networks for Noisy Image Classification
Ricci, S., Uricchio, T., & Bimbo, A. D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-06430-2_37
“Recent methods tried to address the problem by using curriculum learning [4], directly estimating ar X iv :2 21 1.”
MethodPlaat, A. (2022) · Springer eBooks · doi:10.1007/978-981-19-0638-1_6
DeepSTAPLE: Learning to Predict Multimodal Registration Quality for Unsupervised Domain Adaptation
Weihsbach, C., Bigalke, A., Kruse, C. N., Hempe, H., & Heinrich, M. P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-11203-4_5
“…learning to estimate label noise (i.e. difficulty) and guide the optimization process accordingly [4, 22] but the techniques have not been used in the context of noise introduced through registered pixel-wise labels [3,4,11,22,29] or employ more specialized and complex pipelines [5,6,15].”
MethodAn Advanced NMF-Based Approach for Single Cell Data Clustering
Zhao, P., Sheng, Y., & Zhan, X. (2022) · International Conference on Information Communication and Software Engineering (ICICSE) · doi:10.1109/icicse55337.2022.9828919
SCRMA: Snake-Like Robot Curriculum Rapid Motor Adaptation
Liu, X., Sheng, F., & Guo, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13822-5_16
Metadata
- Authors
- Bengio, Y., Louradour, J., Collobert, R., & Weston, J.
- Affiliations
- Université de Montréal (CA)
- NEC Laboratories America, Princeton, NJ
- ORCID
- Bengio, Yoshua
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 41-48
- OpenAlex
- W2296073425
- Semantic Scholar
- 8de174ab5419b9d3127695405efd079808e956e8
- Cited by
- 5,366
- References
- 40
- Citations by year
- 2026: 526 · 2025: 636 · 2024: 551 · 2023: 591 · 2022: 536 · 2021: 711 · 2020: 565 · 2019: 442 · 2018: 293 · 2017: 198 · 2016: 103 · 2015: 63 · 2014: 41 · 2013: 40 · 2012: 33
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- curriculum learning, training order, non-convex optimization, deep neural network, stochastic neural networks, deterministic neural networks, generalization, convergence rate, numerical continuation, global optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.