Learning with AMIGo: Adversarially Motivated Intrinsic Goals
Open accessThe DOI matches its OpenAlex record; 21 of the 21 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · cited 1×
“Deep Reinforcement Learning (RL) has been tremendously successful and continues to show impressive results on a wide range of tasks [e.g. 25, 41, 42, 46].”
BackgroundAsynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 2×
“Deep Reinforcement Learning (RL) has been tremendously successful and continues to show impressive results on a wide range of tasks [e.g. 25, 41, 42, 46].”
BackgroundBengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“Curriculum learning [3] is widely used in machine learning but the curricula are typically handcrafted which can be time consuming.”
BackgroundGrandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455 · cited 1×
“Deep Reinforcement Learning (RL) has been tremendously successful and continues to show impressive results on a wide range of tasks [e.g. 25, 41, 42, 46].”
BackgroundCuriosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815 · cited 4×
“…et al. [5] which uses a random network to compute a prediction error used as a bonus to reward novel states; ICM is Intrinsic Curiosity Module from Pathak et al. [30], which trains forward and inverse models to learn a latent representation used to compare the predicted and actual next states.”
BackgroundMethodInfluentialIMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., et al. (2018) · arXiv · doi:10.48550/arxiv.1802.01561 · Cited by 1,708
Intrinsic Motivation Systems for Autonomous Mental Development
Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890271 · Cited by 1,126
Benchmarking Deep Reinforcement Learning for Continuous Control
Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1604.06778 · Cited by 1,114 · cited 1×
“However, this success has so far been mostly confined to scenarios with reasonably dense rewards, or to those where a perfect model of the environment can be used for search (for example, in the game of Go [41, 8, 26]).”
BackgroundWhat is intrinsic motivation? A typology of computational approaches
Oudeyer, P.-Y. (2007) · Frontiers in Neurorobotics · doi:10.3389/neuro.12.006.2007 · Cited by 811
Robust adversarial reinforcement learning
Pinto, L., Davidson, J., Sukthankar, R., & Gupta, A. (2017) · arXiv · doi:10.48550/arxiv.1703.02702 · Cited by 507 · cited 1×
“Pinto et al. [31] use an adversarial framework to perturb the environment and induce robustness.”
BackgroundAn analysis of model-based Interval Estimation for Markov Decision Processes
Strehl, A. L., & Littman, M. L. (2008) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2007.08.009 · Cited by 428 · cited 1×
“One prominent form is the use of novelty , which in its simplest form can be estimated with state visitation counts [44] and has been extended to high-dimensional state spaces [2, 5, 27].”
BackgroundCount-Based Exploration with Neural Density Models
Ostrovski, G., Bellemare, M. G., Oord, A. V. D., & Munos, R. (2017) · arXiv · doi:10.48550/arxiv.1703.01310 · Cited by 377 · cited 2×
“One prominent form is the use of novelty , which in its simplest form can be estimated with state visitation counts [44] and has been extended to high-dimensional state spaces [2, 5, 27].”
BackgroundMachado, M. C., Bellemare, M. G., Talvitie, E., Veness, J., Hausknecht, M., & Bowling, M. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5699 · Cited by 359
Leveraging Procedural Generation to Benchmark Reinforcement Learning
Cobbe, K., Hesse, C., Hilton, J., & Schulman, J. (2019) · arXiv · doi:10.48550/arxiv.1912.01588 · Cited by 196 · cited 2×
“As advocated in recent work [7, 51, 36, 20], we evaluate AMIG O for procedurally-generated environments instead of trying to learn to perform a specific grounded task.”
BackgroundMethodIntrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., & Fergus, R. (2017) · arXiv · doi:10.48550/arxiv.1703.05407 · Cited by 149 · cited 2×
“We also compare AMIG O to the authors’ implementation 3 of Asymmetric Self-Play ( ASP ) [45].”
MethodAutomatic Curriculum Learning For Deep RL: A Short Survey
Portelas, R., Colas, C., Weng, L., Hofmann, K., & Oudeyer, P.-Y. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/671 · Cited by 127
Towards Generalization and Simplicity in Continuous Control
Rajeswaran, A., Lowrey, K., Todorov, E., & Kakade, S. (2017) · arXiv · doi:10.48550/arxiv.1703.02660 · Cited by 94
A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning
Zhang, A., Ballas, N., & Pineau, J. (2018) · arXiv · doi:10.48550/arxiv.1806.07937 · Cited by 88
Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
Portelas, R., Colas, C., Hofmann, K., & Oudeyer, P.-Y. (2019) · arXiv · doi:10.48550/arxiv.1910.07224 · Cited by 59 · cited 1×
“…on 6 challenging exploration tasks in procedurally generated environments, that agents trained with AMIG O gradually learn to interact with the environment and solve tasks which are too difficult for state-of-the-art methods, and (iii) we perform an extensive qualitative analysis and ablation study.”
The NetHack Learning Environment
Küttler, H., Nardelli, N., Miller, A. H., Raileanu, R., Selvatici, M., Grefenstette, E., & Rocktäschel, T. (2020) · arXiv · doi:10.48550/arxiv.2006.13760 · Cited by 44
Self-citation
Cited by
Goal-Conditioned Reinforcement Learning: Problems and Solutions
Liu, M., Zhu, M., & Zhang, W. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/770 · Cited by 115
“, 2019] Image X HVF [Nair and Finn, 2020] Image X AMIGo [Campero et al., 2021] Image X GAP [Nair et al.”
BackgroundOpen-Ended Learning Leads to Generally Capable Agents
Open Ended Learning Team, Stooke, A., Mahajan, A., Barros, C., Deck, C., Bauer, J., Sygnowski, J., Trebacz, M., Jaderberg, M., Mathieu, M., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.12808 · Cited by 56
“AMIGo (Campero et al., 2021) also generates a curriculum of goals for the agent but does so by looking at the current number of steps needed by the agent to reach the goal.”
BackgroundJiang, M., Grefenstette, E., & Rocktäschel, T. (2020) · arXiv · doi:10.48550/arxiv.2010.03934 · Cited by 33
Self-citationEmpowerment contributes to exploration behaviour in a creative video game
Brändle, F., Stocks, L. J., Tenenbaum, J. B., Gershman, S. J., & Schulz, E. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01661-2 · Cited by 34
Self-citationExploration via Elliptical Episodic Bonuses
Henaff, M., Raileanu, R., Jiang, M., & Rocktäschel, T. (2022) · arXiv · doi:10.48550/arxiv.2210.05805 · Cited by 22
Self-citation
Metadata
- Authors
- Campero, A., Răileanu, R., Küttler, H., Tenenbaum, J. B., Rocktäschel, T., & Grefenstette, E.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2020-06-22
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2006.12122
- OpenAlex
- W3132970947
- Cited by
- 48
- References
- 21
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.