CoChat
Sign Up
PreprintCited by 1Domain Adaptation and Few-Shot LearningReinforcement Learning in RoboticsMachine Learning and Algorithms

The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.

Connecting Context-specific Adaptation in Humans to Meta-learning

Open access
Dubey, Rachit, Grant, Erin, Luo, Michael, Narasimhan, Karthik, Griffiths, Thomas·2020·arXiv
DOI 10.48550/arxiv.2011.13782arXiv 2011.13782
Open this paper in CoChat — free accountOpen PDF

Abstract

Cognitive control, the ability of a system to adapt to the demands of a task, is an integral part of cognition. A widely accepted fact about cognitive control is that it is context-sensitive: Adults and children alike infer information about a task's demands from contextual cues and use these inferences to learn from ambiguous cues. However, the precise way in which people use contextual cues to guide adaptation to a new task remains poorly understood. This work connects the context-sensitive nature of cognitive control to a method for meta-learning with context-conditioned adaptation. We begin by identifying an essential difference between human learning and current approaches to meta-learning: In contrast to humans, existing meta-learning algorithms do not make use of task-specific contextual cues but instead rely exclusively on online feedback in the form of task-specific labels or rewards. To remedy this, we introduce a framework for using contextual information about a task to guide the initialization of task-specific models before adaptation to online feedback. We show how context-conditioned meta-learning can capture human behavior in a cognitive task and how it can be scaled to improve the speed of learning in various settings, including few-shot classification and low-sample reinforcement learning. Our work demonstrates that guiding meta-learning with task information can capture complex, human-like behavior, thereby deepening our understanding of cognitive control.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2011.13782

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Dubey et al.'s model captures the context-sensitivity of human behavior in a simple but well-studied cognitive control task.

    “They showed that such a model captures 'the context-sensitivity of human behavior in a simple but well-studied cognitive control task.'”

    From Meta-learned models of cognition (2024) · cites this paper as Dubey, Grant, Luo, Narasimhan, and Griffiths (2020)

  • In Dubey et al.'s model, contextual cues determine the initialization of a task-specific neural network that is then trained using model-agnostic meta-learning.

    “In their model, contextual cues determine the initialization of a task-specific neural network that is then trained using model-agnostic meta-learning.”

    From Meta-learned models of cognition (2024) · cites this paper as Dubey, Grant, Luo, Narasimhan, and Griffiths (2020)

  • Dubey et al. implemented contextual cues in the meta-learning framework.

    “Dubey, Grant, Luo, Narasimhan, and Griffiths (2020) implemented this idea in the meta-learning framework.”

    From Meta-learned models of cognition (2024) · cites this paper as Dubey, Grant, Luo, Narasimhan, and Griffiths (2020)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 55

  1. Andreas, J.; and Klein, D. 2015. Alignment-based compositional semantics for instruction following. In Proceedings of the Conference on Empirical Methods in Natural Language Processing .
  2. Andreas, J.; Klein, D.; and Levine, S. 2018. Learning with Latent Language. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) , 21662179.
  3. Artzi, Y.; and Zettlemoyer, L. 2013. Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions. Transactions of the Association for Computational Linguistics 1(1): 49-62.
  4. Bahdanau, D.; Hill, F.; Leike, J.; Hughes, E.; Kohli, P.; and Grefenstette, E. 2019. Learning to Understand Goal Specifications by Modelling Reward. In International Conference on Learning Representations . URL https:/ /openreview.net/forum?id= H1xsSjC9Ym .
  5. Baik, S.; Hong, S.; and Lee, K. M. 2019. Learning to Forget for Meta-Learning. arXiv preprint arXiv:1906.05895 . · arXiv
  6. Barcelo, F.; Escera, C.; Corral, M. J.; and Periáñez, J. A. 2006. Task switching and novelty processing activate a common neural network for cognitive control. Journal of cognitive neuroscience 18(10): 1734-1748.
  7. Bengio, S.; Bengio, Y.; Cloutier, J.; and Gecsei, J. 1992. On the optimization of a synaptic learning rule. In Proceedings of the Conference on Optimality in Artificial and Biological Neural Networks , 6-8.
  8. Botvinick, M. M.; Braver, T. S.; Barch, D. M.; Carter, C. S.; and Cohen, J. D. 2001. Conflict monitoring and cognitive control. Psychological review 108(3): 624.
  9. Botvinick, M. M.; and Cohen, J. D. 2014. The computational and neural basis of cognitive control: charted territory and new frontiers. Cognitive science 38(6): 1249-1285.
  10. Branavan, S.; Zettlemoyer, L. S.; and Barzilay, R. 2010. Reading between the lines: Learning to map high-level instructions to commands. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics , 1268-1277. Association for Computational Linguistics.
  11. Braver, T. S.; Paxton, J. L.; Locke, H. S.; and Barch, D. M. 2009. Flexible neural mechanisms of cognitive control within human prefrontal cortex. Proceedings of the National Academy of Sciences 106(18): 7351-7356.
  12. Cohen, J. D.; Botvinick, M.; and Carter, C. S. 2000. Anterior cingulate and prefrontal cortex: who's in control? Nature neuroscience 3(5): 421.
  13. Collins, A. G. 2018. Learning structures through reinforcement. In Goal-Directed Decision Making , 105-123. Elsevier.
  14. Collins, A. G.; and Frank, M. J. 2013. Cognitive control over learning: Creating, clustering, and generalizing task-set structure. Psychological review 120(1): 190.
  15. Donoso, M.; Collins, A. G.; and Koechlin, E. 2014. Foundations of human reasoning in the prefrontal cortex. Science 344(6191): 1481-1486.
  16. Dosenbach, N. U.; Visscher, K. M.; Palmer, E. D.; Miezin, F. M.; Wenger, K. K.; Kang, H. C.; Burgund, E. D.; Grimes, A. L.; Schlaggar, B. L.; and Petersen, S. E. 2006. A core system for the implementation of task sets. Neuron 50(5): 799-812.
  17. Eckstein, M. K.; and Collins, A. G. E. 2020. Computational evidence for hierarchically-structured reinforcement learning in humans. bioRxiv 731752.
  18. Finn, C.; Abbeel, P.; and Levine, S. 2017. Model-Agnostic MetaLearning for Fast Adaptation of Deep Networks. In Proceedings of the 34th International Conference on Machine Learning (ICML) .
  19. Finn, C.; Xu, K.; and Levine, S. 2018. Probabilistic model-agnostic meta-learning. In Advances in Neural Information Processing Systems , 9516-9527.
  20. Frank, M. J.; and Badre, D. 2012. Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. Cerebral cortex 22(3): 509-526.
  21. Ha, D.; Dai, A.; and Le, Q. V . 2017. HyperNetworks. In Proceedings of the 5th International Conference on Learning Representations (ICLR) .
  22. Harrison, B.; Ehsan, U.; and Riedl, M. O. 2018. Guiding reinforcement learning exploration using natural language. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems , 1956-1958. International Foundation for Autonomous Agents and Multiagent Systems.
  23. He, X.; and Peng, Y. 2017. Fine-grained image classification via combining vision and language. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 5994-6002.
  24. Hu, Z.; Chen, T.; Chang, K.-W.; and Sun, Y. 2019. Few-Shot Representation Learning for Out-Of-Vocabulary Words. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, ACL 2019 .
  25. Ioffe, S.; and Szegedy, C. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 . · arXiv
  26. Kim, J.; and Mooney, R. J. 2013. Adapting Discriminative Reranking to Grounded Language Learning. In ACL (1) , 218-227.
  27. Lan, L.; Li, Z.; Guan, X.; and Wang, P. 2019. Meta reinforcement learning with task embedding and shared policy. arXiv preprint arXiv:1905.06527 . · arXiv
  28. Lee, H. B.; Lee, H.; Na, D.; Kim, S.; Park, M.; Yang, E.; and Hwang, S. J. 2019. Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks. arXiv preprint arXiv:1905.12917 . · arXiv
  29. Lee, Y.; and Choi, S. 2018. Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace. In International Conference on Machine Learning , 2933-2942.
  30. MacKay, M.; Vicol, P.; Lorraine, J.; Duvenaud, D.; and Grosse, R. 2019. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. arXiv preprint arXiv:1903.03088 . · arXiv
  31. MacMahon, M.; Stankiewicz, B.; and Kuipers, B. 2006. Walk the talk: Connecting language, knowledge, and action in route instructions. AAAI 2(6): 4.
  32. Narasimhan, K.; Barzilay, R.; and Jaakkola, T. 2018. Grounding Language for Transfer in Deep Reinforcement Learning. Journal of Artificial Intelligence Research 63: 849-874.
  33. Norouzi, M.; Mikolov, T.; Bengio, S.; Singer, Y.; Shlens, J.; Frome, A.; Corrado, G. S.; and Dean, J. 2013. Zero-shot learning by convex combination of semantic embeddings. arXiv preprint arXiv:1312.5650 . · arXiv
  34. Oreshkin, B.; López, P. R.; and Lacoste, A. 2018. TADAM: Task dependent adaptive metric for improved few-shot learning. In Advances in Neural Information Processing Systems , 719-729.
  35. Pahde, F.; Jähnichen, P.; Klein, T.; and Nabi, M. 2018. Crossmodal Hallucination for Few-shot Fine-grained Recognition. arXiv preprint arXiv:1806.05147 . · arXiv
  36. Perez, E.; Strub, F.; De Vries, H.; Dumoulin, V.; and Courville, A. 2018. Film: Visual reasoning with a general conditioning layer. In Thirty-Second AAAI Conference on Artificial Intelligence .
  37. Pong, V.; Gu, S.; Dalal, M.; and Levine, S. 2018. Temporal difference models: Model-free deep rl for model-based control. arXiv preprint arXiv:1802.09081 . · arXiv
  38. Rakelly, K.; Zhou, A.; Quillen, D.; Finn, C.; and Levine, S. 2019. Efficient off-policy meta-reinforcement learning via probabilistic context variables. arXiv preprint arXiv:1903.08254 . · arXiv
  39. Reed, S.; Akata, Z.; Lee, H.; and Schiele, B. 2016. Learning deep representations of fine-grained visual descriptions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 49-58.
  40. Rothfuss, J.; Lee, D.; Clavera, I.; Asfour, T.; and Abbeel, P. 2019. ProMP: Proximal Meta-Policy Search. In ICLR .
  41. Schaul, T.; Horgan, D.; Gregor, K.; and Silver, D. 2015. Universal value function approximators. In International Conference on Machine Learning (ICML) , 1312-1320.
  42. Schmidhuber, J. 1987. Evolutionary principles in self-referential learning . Ph.D. thesis, Institut für Informatik, Technische Universität München.
  43. Snell, J.; Swersky, K.; and Zemel, R. S. 2017. Prototypical Networks for Few-shot Learning. In Advances in Neural Information Processing Systems (NIPS) 30 .
  44. Sutton, R. S.; and Barto, A. G. 2018. Reinforcement learning: An introduction . MIT press.
  45. Thrun, S. 1998. Lifelong learning algorithms. In Learning to learn , 181-209. Springer.
  46. Todorov, E.; Erez, T.; and Tassa, Y. 2012. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems , 5026-5033. IEEE.
  47. Vinyals, O.; Blundell, C.; Lillicrap, T.; Wierstra, D.; et al. 2016. Matching networks for one shot learning. In Advances in Neural Information Processing Systems (NIPS) 29 , 3630-3638.
  48. Vogel, A.; and Jurafsky, D. 2010. Learning to follow navigational directions. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics , 806-814. Association for Computational Linguistics.
  49. Vuorio, R.; Sun, S.-H.; Hu, H.; and Lim, J. J. 2018. Toward multimodal model-agnostic meta-learning. arXiv preprint arXiv:1812.07172 . · arXiv
  50. Werchan, D. M.; Collins, A. G.; Frank, M. J.; and Amso, D. 2015. 8-month-old infants spontaneously learn and generalize hierarchical rules. Psychological science 26(6): 805-815.
  51. Werchan, D. M.; Collins, A. G.; Frank, M. J.; and Amso, D. 2016. Role of prefrontal cortex in learning and generalizing hierarchical rules in 8-month-old infants. Journal of Neuroscience 36(40): 10314-10322.
  52. Xing, C.; Rostamzadeh, N.; Oreshkin, B. N.; and Pinheiro, P. O. 2019. Adaptive Cross-Modal Few-Shot Learning. CoRR abs/1902.07104. URL http:/ /arxiv.org/abs/1902. 07104 . · arXiv
  53. Yoon, S. W.; Seo, J.; and Moon, J. 2019. Tapnet: Neural network augmented with task-adaptive projection for few-shot learning. arXiv preprint arXiv:1905.06549 . · arXiv
  54. Yu, T.; Quillen, D.; He, Z.; Julian, R.; Hausman, K.; Finn, C.; and Levine, S. 2019. Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning. In Conference on Robot Learning (CoRL) . URL https:/ /arxiv.org/abs/ 1910.10897 .
  55. Zou, H.; Ren, T.; Yan, D.; Su, H.; and Zhu, J. 2019. Reward Shaping via Meta-Learning. arXiv preprint arXiv:1901.09330 . · arXiv

Cited by

Sorted by citations

Metadata

Authors
Dubey, R., Grant, E., Luo, M., Narasimhan, K., & Griffiths, T.
Affiliations
Princeton University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2011.13782
arXiv
2011.13782
OpenAlex
W3108142528
Cited by
1
References
2
Citations by year
2024: 1
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
Keywords
meta-learning, cognitive control, contextual cues, few-shot classification, online feedback

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review