Building machines that learn and think like people

In the paper

“Humans, after playing just a small number of games over a span of minutes, can understand the game and its goals well enough to perform better than deep networks do after almost a thousand hours of experience”

In the paper

“If humans are able to first watch an expert playing for a few minutes, they can learn even faster”

In the paper

“The learned DQN network is also rather inflexible to changes in its inputs and goals”

In the paper

“Random play achieves a score of 65.2”

In the paper

“after watching videos of expert play on YouTube for just 2 minutes, we found that we were able to reach scores comparable to or better than the human expert reported in Mnih et al. (2015) after at most 15 to 20 minutes of total practice”

What the sources say

• Not checked yetcites Mnih et al. (2015)

In the paper

“they can learn to play better than chance within a few minutes”

In the paper

“in which people substantially outperform current DQN approaches”

In the paper

“even non-professional humans can grasp the basics of the game after just a few minutes of play”

In the paper

“a physics-engine reconstruction of a tower of wooden blocks from the game Jenga can be used to predict whether (and how) a tower will fall, finding close quantitative fits to how adults make these predictions”

What the sources say

• Not checked yetcites Battaglia et al. 2013

In the paper

“It has been shown to generalize, but also only in limited ways (e.g., from towers of two and three cubes to towers of four cubes)”

In the paper

“Causality is an important factor in the model's success in classifying and generating new examples after seeing just a single example of a new concept (Lake et al. 2015a) (Fig. 5B).”

What the sources say

• Not checked yetcites Lake et al. 2015a

In the paper

“These tasks include generating new examples (Figs.1A-ii and 5B), parsing objects into their essential components (Fig. 1A-iii), and generating new concepts in the style of a particular alphabet (Fig. 1A-iv).”

In the paper

“Deep neural networks have at least a limited notion of compositionality.”

In the paper

“These tasks include generating new examples (Figs.1A-ii and 5B), parsing objects into their essential components (Fig. 1A-iii), and generating new concepts in the style of a particular alphabet (Fig. 1A-iv).”

In the paper

“In contrast, the generative model for characters using BPL does resemble the steps of writing, although even more causally faithful models are possible.”

In the paper

“These tasks include generating new examples (Figs.1A-ii and 5B), parsing objects into their essential components (Fig. 1A-iii), and generating new concepts in the style of a particular alphabet (Fig. 1A-iv).”

In the paper

“For the BPL of learning handwritten characters, causality is operationalized by treating concepts as motor programs, or abstract causal descriptions of how to produce examples of the concept, rather than concrete configurations of specific muscles (Fig. 5A).”

In the paper

“This algorithm can then learn new games faster than a randomly initialized DQN: Scores that might have taken 4 or 5 million frames of learning to reach might now be reached after 1 or 2 million frames of practice.”

What the sources say

• Not checked yetcites Parisotto et al. 2016

In the paper

“Previously learned primitive actions and larger generative pieces can be re-used and re-combined to define new generative models for new characters (Fig. 5A).”

In the paper

“But to perform well, current neural network approaches require much more pre-training than do people or our Bayesian program learning approach.”

In the paper

“In at least some domains, people may not have an especially clever solution to this problem, instead grappling with the full combinatorial complexity of theory learning (Ullman et al. 2012b).”

What the sources say

• Not checked yetcites Ullman et al. 2012b

In the paper

“BPL transfers readily to new concepts because it learns about object parts, sub-parts, and relations, capturing learning about what each concept is like and what concepts are like in general.”

In the paper

“For BPL, pre-training with characters in only five alphabets (for around 150 character types in total) is sufficient to perform human-level one-shot classification and generation of new examples.”

In the paper

“Consistent with the idea of cooperation between learning systems, a recent experiment demonstrated that model-based behavior becomes automatic over the course of training (Economides et al. 2015).”

What the sources say

• Not checked yetcites Economides et al. 2015

In the paper

“these claims usually pertain to the cellular and synaptic levels, with few connections made to systems-level neuroscience and subcortical brain organization”

What the sources say

• Not checked yetcites Edelman 2015

In the paper

“In recent experiments, Tsividis et al. (2015) found that children can use high-level abstract features of a domain to guide hypothesis selection, by reasoning about distributional properties like the ratio of seeds to flowers, and dynamical properties like periodic or monotonic relationships between causes and effects (see also Magid et al. 2015).”

What the sources say

• Not checked yetcites Tsividis et al. 2015
• Not checked yetcites Magid et al. 2015

In the paper

“backpropagation has been of obviously great value in engineering today's best pattern recognition systems.”

In the paper

“The NTM has been trained to perform sequence-to-sequence prediction tasks such as sequence copying and sorting”

What the sources say

• Not checked yetcites Graves et al. 2014

In the paper

“people need to see just one or a few examples of a new concept before being able to recognize new examples, generate new examples, and generate new concepts based on related ones”

What the sources say

• Not checked yetcites Fig. 1A

In the paper

“neural programmer-interpreters learn to represent and execute algorithms such as addition and sorting from fewer examples, by observing inputoutput pairs (like the NTM and DNC), as well as execution traces”

What the sources say

• Not checked yetcites Reed & Freitas 2016

In the paper

“the incorporation of attention has led to substantial performance gains in a variety of domains, including in machine translation (Bahdanau et al. 2015), object recognition (Mnih et al. 2014), and image caption generation (Xu et al. 2015).”

What the sources say

• Not checked yetcites Bahdanau et al. 2015
• Not checked yetcites Mnih et al. 2014
• Not checked yetcites Xu et al. 2015

In the paper

“in DL, the internal structure, or “model,” accumulates at later layers, and is effectively constructing “scaffolds” over the learning process that are then used to train subsequent layers”

What the sources say

• Not checked yetcites Caglar & Hanson 2016
• Not checked yetcites Saxe 2013

In the paper

“during caption generation, the attentional window has been shown to track the objects as they are mentioned in the caption”

What the sources say

• Not checked yetcites Xu et al. 2015

In the paper

“it was only recently that a computer program -AlphaGo -first beat a world-class player (Chouard 2016) by using a combination of deep convolutional neural networks (ConvNets) and Monte-Carlo Tree Search (Silver et al. 2016).”

What the sources say

• Not checked yetcites Chouard 2016
• Not checked yetcites Silver et al. 2016

In the paper

“In generative neural network models, attention has been used to concentrate on generating particular regions of the image rather than the whole image at once”

What the sources say

• Not checked yetcites Gregor et al. 2015

In the paper

“For example, if you are carrying a number of small creatures in a closed steel box, you can predict that as long as the box remains completely closed, the creatures will remain inside.This prediction can be made without knowing anything about the creatures and the way they move, without knowing the initial positions or shapes of the box or the creatures, and without knowing the trajectory of the”

In the paper

“Third, there is extensive evidence that in many cases where the actual physics is simple, humans make large, systematic errors.”

In the paper

“For example, Chinese-American children are more sensitive to whether there is consensus about a behavior or information than Euro-American children (Corriveau & Harris 2010;Corriveau et al. 2013;DiYanni et al. 2015).”

What the sources say

• Not checked yetcites Corriveau & Harris 2010
• Not checked yetcites Corriveau et al. 2013
• Not checked yetcites DiYanni et al. 2015

In the paper

“Such cultural differences in attending to social information in observational learning situations persist into adulthood (Mesoudi et al. 2015).”

What the sources say

• Not checked yetcites Mesoudi et al. 2015

In the paper

“Culturally specific ethno-theories of how children learn (Clegg et al. 2017;Corriveau et al. 2013;Harkness et al. 2007;Super & Harkness 2002) and the learning opportunities to which children have access (Kline 2015;Rogoff 2003) shape their ability to learn through observation.”

What the sources say

• Not checked yetcites Clegg et al. 2017
• Not checked yetcites Corriveau et al. 2013
• Not checked yetcites Harkness et al. 2007

In the paper

“The best findings from such work, although far from final reduction-to-practice, arguably involve the appearance of crude or primitive machine personalities and identities from socially learned intra-/interpersonal relationships possessing cognitiveemotional valences.”

In the paper

“By pre-school, children from non-WEIRD cultures where observational learning is expected and socialized outperform children from WEIRD cultures in observational learning tasks (Correa-Chávez & Rogoff 2009;Mejía-Arauz et al. 2005).”

What the sources say

• Not checked yetcites Correa-Chávez & Rogoff 2009
• Not checked yetcites Mejía-Arauz et al. 2005

In the paper

“Research suggests that from early infancy, children display a core set of abilities that shape their reasoning about the world, including reasoning about both inanimate objects (intuitive physics [e.g., Spelke 1990]) and animate social beings (intuitive psychology [e.g., Dennett 1987;Meltzoff & Moore 1995]).”

What the sources say

• Not checked yetcites Spelke 1990
• Not checked yetcites Dennett 1987
• Not checked yetcites Meltzoff & Moore 1995

In the paper

“This situation also draws on humans' ability to learn by observing others and, like the development of false-belief understanding, children's ability to learn through observation as well as through verbal testimony, which is heavily influenced by sociocultural inputs (Harris 2012).”

What the sources say

• Not checked yetcites Harris 2012

In the paper

“Moreover, research that has examined children's intuitive theories in different cultural settings has suggested the potential for both cross-cultural continuity and variation in their development.”

In the paper

“As early as late infancy, sociocultural inputs such as how parents direct children's attention, or the typical structure of parent-child interaction, may lead to differences in the way children attend to events for the purpose of observational learning (Chavajay & Rogoff 1999).”

What the sources say

• Not checked yetcites Chavajay & Rogoff 1999

In the paper

“Human reasoners, by contrast, can deal with information that is radically incomplete.”

In the paper

“It is clear the DL nets are not normally trained with anything like the experiences had by the developing child, whose learning is based on broad, multisensory experience and is cumulative, with new motor and cognitive skills building on old (Vygotsky 1978).”

What the sources say

• Not checked yetcites Vygotsky 1978

In the paper

“However, similar reported strategies for machine architectures, algorithms, and performance demonstrate only marginal success when used as protocols to reach nearer cognitive-emotional humanness in trending social robot archetypes”

What the sources say

• Not checked yetcites Arbib & Fellous 2004
• Not checked yetcites Asada 2015
• Not checked yetcites Berdahl 2010

In the paper

“Even simplistic artificial cognitive-emotional profiles and personalities thus effect varying control over acquisition and lean of machine domain-general/-specific knowledge, perception and expression of flat or excessive machine affect, and rationality and use of inferential machine attitudes/opinions/beliefs”

What the sources say

• Not checked yetcites Arbib & Fellous 2004
• Not checked yetcites Asada 2015
• Not checked yetcites Berdahl 2010

In the paper

“Conversely, people are capable of predicting characteristics of an end state for problems where it is impossible to predict the intermediate states in detail, as the example of the creatures in the box illustrates.”

In the paper

“The causal processes that generated each of these are very different; and yet a person familiar with cats will recognize any of these depictions, even if they know little of the causal processes underlying sculpture or the formation of clouds.”

In the paper

“For example, a model that combines analogy with visual processing of relational representations has achieved human-level performance on Raven's Progressive Matrices test (Lovett & Forbus 2017).”

What the sources say

• Not checked yetcites Lovett & Forbus 2017

In the paper

“Current distributed representations have difficulty handling even one or two relations.”

In the paper

“the Companion cognitive architecture can use rich relational representations and analogy to perform distant transfer.”

In the paper

“Further, it progresses from simple to complex models in an order that corresponds to the order found in children (Friedman et al. 2010).”

What the sources say

• Not checked yetcites Friedman et al. 2010

In the paper

“Second, qualitative representations combined with analogy been used to model aspects of conceptual change.”

In the paper

“Further, the results from adversarial training indicate that deep learning systems do not construct human-like intermediate representations (Goodfellow et al. 2015; see also target article).”

What the sources say

• Not checked yetcites Goodfellow et al. 2015

In the paper

“People frequently draw inferences from analogous situations, sometimes without awareness of doing so (Day & Gentner 2007).”

What the sources say

• Not checked yetcites Day & Gentner 2007

In the paper

“Learning games with a previously learned analogous game led to more rapid learning than learning without such an analog (Hinrichs & Forbus 2011).”

What the sources say

• Not checked yetcites Hinrichs & Forbus 2011

In the paper

“Simulating humans on these tasks requires between 10 and 100 relations per example.”

In the paper

“For example, on the link plausibility task, in which simple knowledge bases (Freebase, WordNet) are analyzed so that the plausibility of new queries can be estimated (e.g., Is Barack Obama Kenyan?), a combination of analogy and structured logistic regression achieved state-of-the-art performance, with orders of magnitude fewer training examples than distributed representation systems (Liang & For”

What the sources say

• Not checked yetcites Liang & Forbus 2015

In the paper

“Moreover, computational models of structure mapping's matching, retrieval, and generalization operations have been used to simulate a wide range of phenomena, including geometric analogies, transfer learning during problem solving, and moral decision making (Forbus et al. 2017).”

What the sources say

• Not checked yetcites Forbus et al. 2017

In the paper

“Human intuitive knowledge concerns relationships such as”

In the paper

“For example, using a series of sketches to depict motion, a Companion learns intuitive models of force.”

In the paper

“Experiments suggest that human children differ from chimpanzees primarily with respect to social knowledge (Herrmann et al. 2007;2010).”

What the sources say

• Not checked yetcites Herrmann et al. 2007
• Not checked yetcites Herrmann et al. 2010

In the paper

“There is abundant evidence from both children and adults that structure mapping (Gentner 1983) is a major route to acquiring and using knowledge.”

What the sources say

• Not checked yetcites Gentner 1983

In the paper

“recently we showed that it can be used for learning of compositional features (Lázaro-Gredilla et al. 2016).”

What the sources say

• Not checked yetcites Lázaro-Gredilla et al. 2016

In the paper

“These models have achieved remarkable gains in many domains spanning object recognition, speech recognition, and control (LeCun et al. 2015;Schmidhuber 2015).”

What the sources say

• Not checked yetcites LeCun et al. 2015
• Not checked yetcites Schmidhuber 2015

In the paper

“The sophistication of human-like reinforcement learning has yet to be realized in AI systems”

In the paper

“Despite their biological inspiration and performance achievements, these systems differ from human intelligence in crucial ways.”

In the paper

“These types of inferences further accelerate the learning of new tasks.”

In the paper

“From this perspective, the earlypresent capacities for intuitive physics and psychology are also causal models of the world.”

In the paper

“Although humans and neural networks may perform equally well on the MNIST digit recognition task and other large-scale image classification tasks, it does not mean that they learn and think in the same way.”

In the paper

“The best results achieved using deep convolutional nets are very close to human-level performance at an error rate of 0.2% (Ciresan et al. 2012).”

What the sources say

• Not checked yetcites Ciresan et al. 2012

In the paper

“By learning to recognize patterns in these inferences, the outputs of inference can be predicted without having to go through costly intermediate steps.”

In the paper

“Moreover, researchers have trained generic networks to perform structured and even strategic tasks, such as the recent work on using a Deep Q-learning Network (DQN) to play simple video games (Mnih et al. 2015)”

What the sources say

• Not checked yetcites Mnih et al. 2015

In the paper

“Training large-scale relatively generic networks is also the best current approach for object recognition (He et al. 2016;Krizhevsky et al. 2012;Russakovsky et al. 2015;Szegedy et al. 2014)”

What the sources say

• Not checked yetcites He et al. 2016
• Not checked yetcites Krizhevsky et al. 2012
• Not checked yetcites Russakovsky et al. 2015

In the paper

“Similarly, recent results applying convolutional nets to the far more challenging ImageNet object recognition benchmark have shown that human-level performance is within reach on that data set as well (Russakovsky et al. 2015).”

What the sources say

• Not checked yetcites Russakovsky et al. 2015

In the paper

“Mnih et al. (2015) combined ideas from deep learning and reinforcement learning to make a”

What the sources say

• Not checked yetcites Mnih et al. 2015

In the paper

“We suggest that compositionality and learning-to-learn are ingredients that make this type of rapid model learning possible”

In the paper

“It is also important to clarify that the PDP perspective is compatible with”

What the sources say

• Not checked yetcites Rumelhart et al. 1986b
• Not checked yetcites Bottou [2014]

In the paper

“Infants have primitive object concepts that allow them to track objects over time and to discount physically implausible trajectories.”

What the sources say

• Not checked yetcites Wellman & Gelman [1992]

In the paper

“Neural network models and the PDP approach offer a view of the mind (and intelligence more broadly) that is sub-symbolic and often populated with minimal constraints and inductive biases to guide learning.”

In the paper

“It did, however, have particular trouble with Frostbite and other games that required temporally extended planning strategies.”

What the sources say

• Not checked yetcites Mnih et al. 2015

In the paper

“The network was trained to play 49 classic Atari games, proposed as a test domain for reinforcement learning (Bellemare et al. 2013), impressively achieving human-level performance or above on 29 of the games.”

What the sources say

• Not checked yetcites Bellemare et al. 2013
• Not checked yetcites Mnih et al. 2015

In the paper

“But they require a lot of experience to reach this level.”

In the paper

“the game of Frostbite provides incremental rewards for reaching each active ice floe, providing the DQN with the relevant sub-goals for completing the larger task of building an igloo”

In the paper

“At the age of 2 months, and possibly earlier, human infants expect inanimate objects to follow principles of persistence, continuity, cohesion, and solidity”

What the sources say

• Not checked yetcites Spelke 1990
• Not checked yetcites Spelke et al. 1995

In the paper

“The DQN learns to play Frostbite and other Atari games by combining a powerful pattern recognizer (a deep convolutional neural network) and a simple model-free reinforcement learning algorithm (Q-learning [Watkins & Dayan 1992]).”

What the sources say

• Not checked yetcites Watkins & Dayan 1992

In the paper

“Although it is interesting that the DQN learns to play games at human-level performance while assuming very little prior knowledge, the DQN may be learning to play Frostbite and other games in a very different way than people do.”

In the paper

“Human learnersunlike DQN and many other deep learning systemsapproach new problems armed with extensive prior experience”

In the paper

“Additionally, the DQN incorporates experience replay, where each of these frames is replayed approximately eight more times on average over the course of learning.”

What the sources say

• Not checked yetcites Mnih et al. 2015

In the paper

“In Frostbite, it is possible to figure out the higher-level goal of building an igloo without incremental feedback”

In the paper

“Moreover, people learn more than how to do pattern recognition: they learn a concept, that is, a model of the class that allows their acquired knowledge to be flexibly applied in new ways.”

In the paper

“Young children have a rich knowledge of intuitive physics”

In the paper

“sparse feedback is a source of difficulty in other Atari 2600 games such as Montezuma's Revenge”

In the paper

“In Mnih et al. (2015), the DQN was compared with a professional gamer who received approximately 2 hours of practice on each of the 49 Atari games (although he or she likely had prior experience with some of the games).”

What the sources say

• Not checked yetcites Mnih et al. 2015

In the paper

“The underlying cognitive representations can be understood as”

What the sources say

• Not checked yetcites Carey 2004
• Not checked yetcites Gopnik et al. 2004
• Not checked yetcites Wellman & Gelman 1992

In the paper

“Early in development, humans have a foundational understanding of several core domains”

What the sources say

• Not checked yetcites Spelke 2003
• Not checked yetcites Spelke & Kinzler 2007

In the paper

“people require little or no retraining or reconfiguration, adding new tasks and goals to their repertoire with relative ease”

In the paper

“people likely do not rely on incremental scoring in the same way when figuring out how to play a new game”

In the paper

“People learn a lot more from a lot less, and capturing these human-level learning abilities in machines is the Characters Challenge.”

In the paper

“each domain is organized by a set of entities and abstract principles relating the entities to each other”

In the paper

“Success in this game requires a temporally extended plan to ensure the agent can accomplish a sub-goal (such as reaching an ice floe) and then safely proceed to the next sub-goal.”

In the paper

“With the full 924 hours of unique experience and additional replay, the DQN achieved less than 10% of human-level performance during a controlled test session (see DQN in Fig. 3).”

What the sources say

• Not checked yetcites Mnih et al. 2015

In the paper

“Changing the color or appearance of objects or changing the goals of the network would have devastating consequences on performance if the network is not retrained”

In the paper

“Without these sub-goals, the DQN would have to take random actions until it accidentally builds an igloo and is rewarded for completing the entire level”

In the paper

“These additional abilities come for free along with the acquisition of the underlying concept.”

In the paper

“Even for these simple visual concepts, people are still better and more sophisticated learners than the best algorithms for character recognition.”

In the paper

“Although both the original DQN and these more recent variants require multiple hours of experience to perform reliably better than random play”

What the sources say

• Not checked yetcites Mnih et al. (2015)
• Not checked yetcites Schaul et al. 2016
• Not checked yetcites Wang et al. 2016

In the paper

“Deep neural networks often need more data than people do to solve the same types of problems”

In the paper

“One alternative to a cue-based account is to use generative models of action choice, as in the Bayesian inverse planning, or Bayesian theory of mind (ToM), models of Baker et al. (2009) or the naive utility calculus models of Jara-Ettinger et al. (2015)”

What the sources say

• Not checked yetcites Baker et al. 2009
• Not checked yetcites Jara-Ettinger et al. 2015

In the paper

“altering the setting of an agent or their goals and beliefs is difficult to reason about without understanding intuitive psychology”

In the paper

“Each of these physical judgments may require new features or new training for a pattern recognition account to work at the same level as the model-based simulator”

In the paper

“When learning the meanings of words in their native language, children make meaningful generalizations from very sparse data”

What the sources say

• Not checked yetcites Carey & Bartlett 1978
• Not checked yetcites Landau et al. 1988
• Not checked yetcites Markman 1989

In the paper

“a simple inductive bias, for example, the tendency to notice things that move other things, can bootstrap reasoning about more abstract concepts of agency”

What the sources say

• Not checked yetcites Ullman et al. 2012a

In the paper

“Intuitive psychology provides a basis for efficient learning from others, especially in teaching settings with the goal of communicating knowledge efficiently”