The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Teacher–Student Curriculum Learning
Open accessAbstract
We propose Teacher-Student Curriculum Learning (TSCL), a framework for automatic curriculum learning, where the Student tries to learn a complex task, and the Teacher automatically chooses subtasks from a given set for the Student to train on. We describe a family of Teacher algorithms that rely on the intuition that the Student should practice more those tasks on which it makes the fastest progress, i.e., where the slope of the learning curve is highest. In addition, the Teacher algorithms address the problem of forgetting by also choosing tasks where the Student's performance is getting worse. We demonstrate that TSCL matches or surpasses the results of carefully hand-crafted curricula in two tasks: addition of decimal numbers with long short-term memory (LSTM) and navigation in Minecraft. Our automatically ordered curriculum of submazes enabled to solve a Minecraft maze that could not be solved at all when training directly on that maze, and the learning was an order of magnitude faster than a uniform sampling of those submazes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · cited 1×
“For learning, we used the proximal policy optimization (PPO) algorithm [39] implemented using Keras [37] and optimized for real-time environments.”
MethodSelf-citationWatkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 4×
“One approach to overcome this problem is to use curriculum learning [6]–[9], where tasks are ordered by increasing difficulty and training only proceeds to harder tasks once easier ones are mastered.”
BackgroundInfluentialPlanning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
Bellman, R. (1957) · Indiana University Mathematics Journal · doi:10.1512/iumj.1957.6.56038 · Cited by 2,408
Hybrid computing using a neural network with dynamic external memory
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., et al. (2016) · Nature · doi:10.1038/nature20101 · Cited by 1,412 · cited 1×
“More recent results include learning to execute short programs [7], finding shortest paths in graphs [8], and learning to play first-person shooter [9].”
ResultEnd-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Intrinsic Motivation Systems for Autonomous Mental Development
Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890271 · Cited by 1,126
What is intrinsic motivation? A typology of computational approaches
Oudeyer, P.-Y. (2007) · Frontiers in Neurorobotics · doi:10.3389/neuro.12.006.2007 · Cited by 811
Formal theory of creativity, fun, and intrinsic motivation (1990–2010)
Schmidhuber, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2056368 · Cited by 747
Automatic Goal Generation for Reinforcement Learning AgentsPreprint
Held, D., Geng, X.-Y., Florensa, C., & Abbeel, P. (2017) · International Conference on Machine Learning · doi:10.48550/arxiv.1705.06366 · Cited by 622
Curious model-building control systems
Schmidhuber, J. (1991) · [Proceedings] 1991 IEEE International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1991.170605 · Cited by 541
VIME: Variational Information Maximizing Exploration
Houthooft, R., Chen, X., Duan, Y., Schulman, J., Turck, F. D., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09674 · Cited by 472
Self-citationActive learning of inverse models with intrinsically motivated goal exploration in robots
Baranes, A., & Oudeyer, P.-Y. (2013) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2012.05.008 · Cited by 455
Neural Programmer-Interpreters
Reed, S., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06279 · Cited by 347 · cited 1×
“Decimal addition has also been explored in (Kalchbrenner et al., 2015; Reed and De Freitas, 2015; Kaiser and Sutskever, 2015), sometimes improving results over original work in (Zaremba and Sutskever, 2014).”
ResultA Deep Hierarchical Approach to Lifelong Learning in Minecraft
Tessler, C., Givony, S., Zahavy, T., Mankowitz, D., & Mannor, S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10744 · Cited by 299
Multi-Modal Curriculum Learning for Semi-Supervised Image Classification
Gong, C., Tao, D., Maybank, S. J., Liu, W., Kang, G., & Yang, J. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2563981 · Cited by 285
Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models
Stadie, B. C., Levine, S., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.00814 · Cited by 280
Kalchbrenner, N., Danihelka, I., & Graves, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.01526 · Cited by 267
Kaiser, Ł., & Sutskever, I. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.08228 · Cited by 214
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., & Fergus, R. (2017) · arXiv · doi:10.48550/arxiv.1703.05407 · Cited by 149 · cited 2×
“[22] proposed a method to generate incremental goals and, therefore, curricula automatically.”
MethodLabel Propagation via Teaching-to-Learn and Learning-to-Teach
Gong, C., Tao, D., Liu, W., Liu, L., & Yang, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2514360 · Cited by 138
Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning
Wu, Y., & Tian, Y. (2017) · International Conference on Learning Representations · Cited by 136
Multi-Armed Bandits for Intelligent Tutoring SystemsPreprint
Clement, B., Roy, D., Oudeyer, P., & Lopes, M. C. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1310.3174 · Cited by 127
Efficient Exploration in Reinforcement Learning
Langford, J. C. (2010) · Encyclopedia of Machine Learning · doi:10.1007/978-0-387-30164-8_244 · Cited by 66
The strategic student approach for life-long exploration and learning
Lopes, M. C., & Oudeyer, P. (2012) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2012.6400807 · Cited by 56
Progressive Stochastic Learning for Noisy Labels
Han, B., Tsang, I. W.-H., Chen, L., Yu, C. P., & Fung, S.-F. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2792062 · Cited by 36
Ensemble Teaching for Hybrid Label Propagation
Gong, C., Tao, D., Chang, X., & Yang, J. J. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2773562 · Cited by 34
Learning to Multi-Task by Active SamplingPreprint
Sharma, S., Jha, A., Hegde, P., & Ravindran, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.06053 · Cited by 33
Faster Reinforcement Learning Using Active SimulatorsPreprint
Jain, V., & Tulabandhula, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07853 · Cited by 2
Cited by
Learning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Science Robotics · doi:10.1126/scirobotics.abc5986 · Cited by 1,156
“Instead of directly using the reward function to evaluate the learning progress [25, 41–43], we evaluate c T s by the traversability of generated terrains, which is defined as the success rate of traversing a terrain.”
Human-in-the-loop machine learning: a state of the art
Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., & Fernández-Leal, Á. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10246-w · Cited by 1,025
Learning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000448343 · Cited by 752
A Survey on Curriculum Learning
Wang, X., Chen, Y., & Zhu, W. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3069908 · Cited by 692
Solving Rubik's Cube with a Robot Hand
OpenAI, Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B., Petron, A., Paino, A., Plappert, M., Powell, G., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.07113 · Cited by 602
Metadata
- Authors
- Matiisen, T., Oliver, A., Cohen, T., & Schulman, J.
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2020-09
- Volume
- 31
- Issue
- 9
- Pages
- 3732-3740
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2731828881
- Cited by
- 320
- References
- 35
- FWCI
- 17.96
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.