The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index; 22 of the 52 sources its claims cite were checked for support.
Human-like systematic generalization through a meta-learning neural network
Open accessAbstract
The power of human language and thought arises from systematic compositionality—the algebraic ability to understand and produce novel combinations from known components. Fodor and Pylyshyn 1 famously argued that artificial neural networks lack this capacity and are therefore not viable models of the mind. Neural networks have advanced considerably in the years since, yet the systematicity challenge persists. Here we successfully address Fodor and Pylyshyn’s challenge by providing evidence that neural networks can achieve human-like systematicity when optimized for their compositional skills. To do so, we introduce the meta-learning for compositionality (MLC) approach for guiding training through a dynamic stream of compositional tasks. To compare humans and machines, we conducted human behavioural experiments using an instruction learning paradigm. After considering seven different models, we found that, in contrast to perfectly systematic but rigid probabilistic symbolic models, and perfectly flexible but unsystematic neural networks, only MLC achieves both the systematicity and flexibility needed for human-like generalization. MLC also advances the compositional skills of machine learning systems in several systematic generalization benchmarks. Our results show how a standard neural network architecture, optimized for its compositional skills, can mimic human systematic generalization in a head-to-head comparison.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper introduces meta-learning for compositionality (MLC), an optimization procedure that trains a standard transformer over a dynamic stream of few-shot compositional tasks. In head-to-head human experiments and machine-learning benchmarks, MLC achieves human-like systematic generalization and reproduces human-like inductive biases and error patterns, though it still fails on some productivity and structural-generalization challenges.
Key findings
- On the few-shot instruction-learning task, the most systematic MLC run achieved 100% exact-match accuracy; when sampling, MLC produced systematic outputs at 82.4% versus 80.7% for humans, and handled longer output sequences at 77.8% versus 72.5% for humans.
- MLC reproduced human-like errors: one-to-one translation errors were 56.3% of MLC errors versus 24.4% for people, and iconic-concatenation errors involving function 3 were 13.8% versus 23.3% for people.
- In the open-ended task, MLC matched the modal human response in 65.0% of samples; across samples, 66.0% followed one-to-one (62.1% for people), 85.0% followed iconic concatenation (79.3% for people), and 99.0% chose a unique response for each unique command (93.1% for people).
- On benchmarks, MLC reached error rates of 0.22% on SCAN “add jump,” 0.04% on “around right,” 0.06% on “opposite right,” and 0.87% on COGS lexical generalization; however, it failed SCAN length and three COGS structural splits with 100% error.
Methods
- Human behavioural experiments were run on Amazon Mechanical Turk using a pseudolanguage instruction-learning paradigm: n = 25 participants completed a few-shot task with 14 study instructions and 10 query instructions, and n = 29 different participants completed an open-ended task with seven unknown instructions.
- MLC used a standard seq2seq transformer with 3 encoder/decoder layers, 8 attention heads per layer, embeddings of size 128, feedforward hidden size 512, and about 1.4 million parameters; it was meta-trained over episodes generated from randomly generated interpretation grammars, with frozen weights at test.
- MLC variants included algebraic-only, copy-only, behaviourally informed, and joint training; benchmark experiments used SCAN and COGS with surface-level word-type permutations and a modular procedure for long in-context sequences, scoring exact-match error rates.
Limitations
- The paper states that MLC does not automatically handle unpractised forms of generalization or concepts outside the meta-learning distribution, fails SCAN length and three COGS structural splits at 100% error, lacks a mechanism for emitting new symbols, and is untested on the full complexity of natural language and other modalities.
- The paper also states that optimizing over many related grammar-based tasks is not developmentally plausible, and that MLC does not solve every challenge raised by Fodor and Pylyshyn.
- Evident from the text: the human experiments use modest samples (n = 25 and n = 29) and a pseudolanguage with abstract coloured-circle outputs rather than natural language; some MLC variants are optimized directly on human behavioural data, so human-like error patterns may partly reflect those training targets.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
39 Own result · 26 Own method · 13 No basis given · 11 Own interpretation · 5 Supported · 4 Partly supported · 4 Sources not found · 4 No full text of the sources · 3 Not checked yet · 2 Supported so far · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Notably, modern neural networks still struggle on tests of systematicity11–18—tests that even a minimally algebraic mind should pass2.”
§ Human-like systematic generalization through a meta-learning neural network > Main
What the sources say
- Not matchedcites [11] Lake, B. M. & Baroni, M. Generalization without systematicity: on the compositional skills of sequence-to-sequence recurrent networks. In Proc. International Conference on Machine Learning (ICML) (eds. Dy, J. & Krause, A.) 2873–2882 (PMLR, 2018).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked yetcites [12] Ettinger, A., Elgohary, A., Phillips, C. & Resnik, P. Assessing composition in sentence vector representations. In Proc. 7th International Conference on Computational Linguistics, (COLING 2018) 1790–1801 (Association for Computational Linguistics, 2018).
- Supportedcites [13] Bahdanau, D. et al. CLOSURE: assessing systematic generalization of CLEVR models. In Proc. NAACL Workshop on Visually Grounded Interaction and Language (ViGIL) (2019). source
The Q1 (CLEVR) section says that the evaluated neural models often struggle on CLOSURE questions. Section 6 (Discussion) likewise reports that their performance deteriorates on unfamiliar combinations of known primitives, supporting the claim about tests of systematicity. (LLM, conf=0.93)
- Supportedcites [14] Keysers, D. et al. Measuring compositional generalization: a comprehensive method on realistic data. In Proc. International Conference on Learning Representations (ICLR) (2019). source
The Introduction says state-of-the-art machine-learning methods often fail to capture compositional structure and generalize compositionally. The additional SCAN results give a concrete test: both Transformer models scored 0% on the length split, while the LSTM scored around 14%. These passages support the attributed claim that modern neural networks struggle on tests of systematicity. (LLM, conf=0.92)
- Partly supportedcites [15] Yu, L. & Ettinger, A. Assessing phrasal representation and composition in transformers. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP) 4896–4907 (Association for Computational Linguistics, 2020). sourceExperimental — accuracy not yet measured
The Discussion (p. 8) reports little evidence of sophisticated phrase composition in the tested transformers once word overlap is controlled, supporting a narrower claim that these models struggle on controlled composition tests. It also reports strong performance on classic sense-selection items. The source therefore does not establish that modern neural networks generally struggle on tests of systematicity. (LLM, conf=0.84)
- Supportedcites [16] Kim, N. & Linzen, T. COGS: a compositional generalization challenge based on semantic interpretation. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP) 9087–9105 (2020). source
The Introduction (p. 1) identifies systematicity as a form of compositional generalization. The Results (p. 6) show that the tested Transformers and LSTMs performed well in distribution but poorly on the COGS generalization test, supporting the cited claim. (LLM, conf=0.93)
- Not checked — no full text of the sourcecites [17] Hupkes, D., Dankers, V., Mul, M. & Bruni, E. Compositionality decomposed: how do neural networks generalize? J. Artif. Int. Res. 67, 757–795 (2020). source
- Supportedcites [18] Press, O. et al. Measuring and narrowing the compositionality gap in language models. Preprint at https://arxiv.org/abs/2210.03350 (2022). source
The Introduction reports that language models fail to combine facts in a large fraction of compositional questions, even when they answer the constituent questions correctly. It also reports a compositionality gap of roughly 40% across the model sizes tested, supporting the attributed claim that modern neural networks still struggle on such tests. (LLM, conf=0.90)
- Not matchedcites [2] Marcus, G. F. The Algebraic Mind: Integrating Connectionism and Cognitive Science (MIT Press, 2003).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The tasks from the machine-learning literature that we experimented with, SCAN11,66 and COGS16, feature long sequences as (in-context) study examples.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
What the sources say
- Not matchedcites [11] Lake, B. M. & Baroni, M. Generalization without systematicity: on the compositional skills of sequence-to-sequence recurrent networks. In Proc. International Conference on Machine Learning (ICML) (eds. Dy, J. & Krause, A.) 2873–2882 (PMLR, 2018).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites [66] Loula, J., Baroni, M. & Lake, B. M. Rearranging the familiar: testing compositional generalization in recurrent networks. Preprint at http://arxiv.org/abs/1807.07545 (2018). sourceExperimental — accuracy not yet measured
Section 2 describes SCAN as a sequence-to-sequence task that maps commands to action sequences, and notes a test split with longer action sequences than the training set. This supports the reference to sequences in SCAN, but the paper does not characterize them as long in-context study examples; it describes conventional training and test sets instead. COGS is attached to a separate citation in the citing sentence. (LLM, conf=0.82)
- Partly supportedcites [16] Kim, N. & Linzen, T. COGS: a compositional generalization challenge based on semantic interpretation. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP) 9087–9105 (2020). sourceExperimental — accuracy not yet measured
The COGS paper describes sentence-to-logical-form examples and reports long output sequences, including a training example of 153 tokens (Section 5.2.1, p. 7). It does not present COGS examples as in-context study examples; its models were trained on the dataset (Section 5.1, p. 6). (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“MLC also advances the compositional skills of machine learning systems in several systematic generalization benchmarks.”
- Also stated in: § Machine learning benchmarks · plain
“Beyond predicting human behaviour, MLC can achieve error rates of less than 1% on machine learning benchmarks for systematic generalization.”
§ Human-like systematic generalization through a meta-learning neural network > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Machine learning benchmarks · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“MLC outperforms all alternatives, including the same highly algebraic MLC model as described in the previous experiment (MLC (algebraic only)) and a probabilistic symbolic model that uses the three inductive biases to generate responses but, in contrast to MLC, is not capable of optimizing for other patterns in the human behaviour (Table 1; symbolic (oracle/biases)).”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The algebraic interpretation grammar provides the correct normative standard for the few-shot instruction-learning task.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Across all model samples, 66.0% followed one-to-one (62.1% for people), 85.0% followed iconic concatenation (79.3% for people) and the vast majority (99.0%) chose a unique response for each unique command (93.1% for people).”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The algebraic interpretation grammar provides the correct normative standard for the few-shot instruction-learning task.
In the paper
“People’s responses also followed a third bias related to mutual exclusivity that encourages assigning unique meanings to unique words27.”
§ Human-like systematic generalization through a meta-learning neural network > Behavioural results
What the sources say
- Not checked — no full text of the sourcecites [27] Markman, E. M. & Wachtel, G. F. Children’s use of mutual exclusivity to constrain the meanings of words. Cogn. Psychol. 20, 121–157 (1988). source
plaincorrelationaln = 29- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Human participants recruited via Amazon Mechanical Turk are representative of the broader human population for these compositional skills.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although human few-shot learning behaviour can be well characterized by either MLC or a probabilistic symbolic model, a test of more open-ended behaviour emphasizes MLC’s relative strengths.”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The algebraic interpretation grammar provides the correct normative standard for the few-shot instruction-learning task.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Importantly, a single transformer can be optimized for both the few-shot learning and open-ended instruction tasks (MLC (joint)); in fact, this is the strongest overall model across experiments for predicting human behaviour (additional analysis is shown in Extended Data Fig. 5 and Supplementary Information 1).”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The algebraic interpretation grammar provides the correct normative standard for the few-shot instruction-learning task.
In the paper
“Notably, participants also generalized correctly in 72.5% of cases to longer output sequences than seen during training (an example is shown as the last instruction in Fig. 2b (i)), which is a type of generalization that neural networks often struggle with11.”
§ Human-like systematic generalization through a meta-learning neural network > Behavioural results
What the sources say
- Not matchedcites [11] Lake, B. M. & Baroni, M. Generalization without systematicity: on the compositional skills of sequence-to-sequence recurrent networks. In Proc. International Conference on Machine Learning (ICML) (eds. Dy, J. & Krause, A.) 2873–2882 (PMLR, 2018).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainn = 25- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The algebraic interpretation grammar provides the correct normative standard for the few-shot instruction-learning task.
- Rests on an assumption: Human participants recruited via Amazon Mechanical Turk are representative of the broader human population for these compositional skills.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The MLC transformer performs comparably to a probabilistic symbolic model that assumes that people infer the gold grammar but respond stochastically with lapses based on the human inductive biases (symbolic (oracle/biases)).”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The algebraic interpretation grammar provides the correct normative standard for the few-shot instruction-learning task.
- Probably not a substantial claim. 43
- It may state two separate claims as one. 38
- The quote only partly states this claim. 1
Every source the paper cites for this claim was checked and supports it.
In the paper
“For COGS, both models were optimized for 300 epochs (also without early stopping), which is slightly more training than the extended amount prescribed in ref. 67 for their strong seq2seq baseline.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
What the sources say
- Supportedcites [67] Csordás, R., Irie, K. & Schmidhuber, J. The devil is in the detail: simple tricks improve systematic generalization of transformers. In Proc. EMNLP 2021—2021 Conference on Empirical Methods in Natural Language Processing 619–634 (Association for Computational Linguistics, 2021). source
Section 3.2 reports training COGS models without early stopping for a fixed 50,000 steps. The dataset statistics in Appendix G give 24,155 training examples, and Table 4 gives a batch size of 128; together these amount to about 265 epochs, slightly fewer than the citing authors’ 300. The source supports the comparison, not the citing authors’ statement about how they trained their own two models. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“First, children are not born with an adult-like ability to compose functions; in fact, there seem to be important changes between infancy58 and pre-school59 that could be tied to learning.”
§ Human-like systematic generalization through a meta-learning neural network > Discussion
What the sources say
- Partly supportedcites [58] Piantadosi, S. T., Palmeri, H. & Aslin, R. Limits on composition of conceptual operations in 9-month-olds. Infancy 23, 310–324 (2018). sourceExperimental — accuracy not yet measured
The Introduction and Conclusion report that 9-month-olds did not predictively compose simple feature-change functions, in contrast to preschoolers. The General discussion says the developmental change remains unexplained and suggests developing abilities as possible contributors. The paper does not test newborns or establish that learning causes the change. (LLM, conf=0.82)
- Partly supportedcites [59] Piantadosi, S. & Aslin, R. Compositional reasoning in early childhood. PLoS ONE 11, e0147734 (2016). sourceExperimental — accuracy not yet measured
The Abstract reports that children aged 3.5–4.5 can compose novel functions, supporting the preschool part of the sentence. The Discussion allows that prior learning may have contributed, but the paper does not establish a change from infancy to preschool or show that children lack this ability at birth. (LLM, conf=0.84)
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“On test episodes, the model weights are frozen and no task-specific parameters are provided32.”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
What the sources say
- Supportedcites [32] Chan, S. C. Y. et al. Data distributional properties drive emergent in-context learning in transformers. In Advances in Neural Information Processing Systems 35 (eds Koyejo, S. et al.) 18878–18891 (Curran Associates, 2022). source
Section 2.3 (p. 4) says trained models are evaluated on sequences whose labels are reassigned for each sequence, so the model must use the current context to predict the query label. Appendix C.4 (p. 16) states that the same model is used across training and evaluation. The described test setup uses the trained model and contextual examples, not task-specific parameters or weight updates. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“These response patterns can be compared to biases in language acquisition more generally; indeed, forms of one-to-one27 and iconic concatenation28,29 are widely attested in natural language.”
§ Human-like systematic generalization through a meta-learning neural network > Behavioural results
What the sources say
- Not checked — no full text of the sourcecites [27] Markman, E. M. & Wachtel, G. F. Children’s use of mutual exclusivity to constrain the meanings of words. Cogn. Psychol. 20, 121–157 (1988). source
- Not checked — no full text of the sourcecites [28] Haiman, J. The iconicity of grammar: isomorphism and motivation. Language 56, 515–540 (1980). source
- Supportedcites [29] de Ruiter, L., Theakston, A., Brandt, S. & Lieven, E. Iconicity affects children’s comprehension of complex sentences: the role of semantics, clause order, input and individual differences. Cognition 171, 202–224 (2018). source
This source is cited for iconic concatenation, not the separately cited one-to-one bias. The Introduction describes clause order mirroring event order as iconic in English and other languages, and Section 1.2.3 reports corpus evidence that iconic orders are frequent for some sentence types. That supports the broad claim that forms of this bias occur in natural language. (LLM, conf=0.87)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“MLC also allows neural networks to tackle other existing challenges, including making systematic use of isolated primitives11,16 and using mutual exclusivity to infer meanings44.”
§ Human-like systematic generalization through a meta-learning neural network > Discussion
What the sources say
- Not matchedcites [11] Lake, B. M. & Baroni, M. Generalization without systematicity: on the compositional skills of sequence-to-sequence recurrent networks. In Proc. International Conference on Machine Learning (ICML) (eds. Dy, J. & Krause, A.) 2873–2882 (PMLR, 2018).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [16] Kim, N. & Linzen, T. COGS: a compositional generalization challenge based on semantic interpretation. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP) 9087–9105 (2020). source
Section 3.1 (p. 4) describes a COGS challenge in which primitives appear only as isolated words in training but must be interpreted in sentences at generalization time. This citation is attached to the isolated-primitives challenge, not to the citing authors’ claim about MLC or to the separately cited mutual-exclusivity challenge. (LLM, conf=0.91)
- Not matchedcites [44] Gandhi, K. & Lake, B. M. Mutual exclusivity as a challenge for deep neural networks. In Proc. Advances in Neural Information Processing Systems (NeurIPS) 33 (eds Larochelle, H. et al.) 14182–14192 (Curran Associates, 2020).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Systematicity continues to challenge models11–18 and motivates new frameworks34–41.”
§ Human-like systematic generalization through a meta-learning neural network > Discussion
What the sources say
- Not matchedcites [11] Lake, B. M. & Baroni, M. Generalization without systematicity: on the compositional skills of sequence-to-sequence recurrent networks. In Proc. International Conference on Machine Learning (ICML) (eds. Dy, J. & Krause, A.) 2873–2882 (PMLR, 2018).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked yetcites [12] Ettinger, A., Elgohary, A., Phillips, C. & Resnik, P. Assessing composition in sentence vector representations. In Proc. 7th International Conference on Computational Linguistics, (COLING 2018) 1790–1801 (Association for Computational Linguistics, 2018).
- Supportedcites [13] Bahdanau, D. et al. CLOSURE: assessing systematic generalization of CLEVR models. In Proc. NAACL Workshop on Visually Grounded Interaction and Language (ViGIL) (2019). source
The Abstract reports that state-of-the-art CLEVR models often lack systematicity on CLOSURE and describes a new Vector-NMN architecture developed to improve systematic generalization. The Discussion likewise reports persistent generalization gaps and proposes combining the strengths of different model types in future work. (LLM, conf=0.90)
- Supportedcites [14] Keysers, D. et al. Measuring compositional generalization: a comprehensive method on realistic data. In Proc. International Conference on Learning Representations (ICLR) (2019). source
The Abstract reports that three machine-learning architectures fail to generalize compositionally. It also says the lack of realistic benchmarks makes improvements difficult to evaluate and introduces a new benchmark-construction method in response, supporting the attributed motivation for a new framework. (LLM, conf=0.90)
- Supportedcites [15] Yu, L. & Ettinger, A. Assessing phrasal representation and composition in transformers. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP) 4896–4907 (Association for Computational Linguistics, 2020). source
The Discussion (p. 8) says that, after controlling for word overlap, the tested transformers show little evidence of sophisticated phrase composition and predicts related weaknesses on future tests. This supports the claim that systematic meaning composition remains a challenge for these models. The separate statement about new frameworks has its own citation range and is not attributed to this source. (LLM, conf=0.88)
- Supportedcites [16] Kim, N. & Linzen, T. COGS: a compositional generalization challenge based on semantic interpretation. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP) 9087–9105 (2020). source
The Conclusion (p. 9) reports that Transformers and LSTMs performed poorly on COGS generalization tests despite near-perfect in-domain performance. It also says the challenge can motivate development of new computational models, supporting the sentence’s broader point. (LLM, conf=0.92)
- Not checked — no full text of the sourcecites [17] Hupkes, D., Dankers, V., Mul, M. & Bruni, E. Compositionality decomposed: how do neural networks generalize? J. Artif. Int. Res. 67, 757–795 (2020). source
- Supportedcites [18] Press, O. et al. Measuring and narrowing the compositionality gap in language models. Preprint at https://arxiv.org/abs/2210.03350 (2022). source
The Abstract reports that larger GPT-3 models showed no corresponding improvement in the compositional reasoning tested, and introduces self-ask to address that gap. The Conclusion likewise describes self-ask as an improvement over chain-of-thought prompting. This supports the sentence's broad point about a continuing model challenge motivating a new approach, although the paper studies compositional reasoning in multi-hop questions rather than systematicity in every setting. (LLM, conf=0.87)
- Supportedcites [34] Collins, A. G. E. & Frank, M. J. Cognitive control over learning: creating, clustering, and generalizing task-set structure. Psychol. Rev. 120, 190–229 (2013). source
The citation is attached to the sentence’s “new frameworks” clause. The Introduction motivates computational models for creating and reusing task-set structure to generalize learned behavior, and the Discussion describes the authors’ new C-TS model for that purpose. The paper is therefore an example of the framework the citing sentence refers to. (LLM, conf=0.85)
- Not matchedcites [35] Chen, X., Liang, C., Yu, A. W., Song, D. & Zhou, D. Compositional generalization via neural-symbolic stack machines. In Proc. Advances in Neural Information Processing Systems 33 (eds Larochelle, H. et al.) 1690–1701 (Curran Associates, 2020).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
Every source the paper cites for this claim was checked and supports it.
In the paper
“As shown in Fig. 4 and detailed in the ‘Architecture and optimizer’ section of the Methods, MLC uses the standard transformer architecture26 for memory-based meta-learning.”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
What the sources say
- Supportedcites [26] Vaswani, A. et al. Attention is all you need. In Proc. Advances in Neural Information Processing Systems 30 (eds Guyon, I. et al.) 5998–6008 (Curran Associates, 2017). source
The citation identifies the transformer architecture used by MLC, rather than attributing MLC's design to Vaswani et al. Section 3 (p. 2) describes the Transformer's encoder–decoder architecture and its stacked self-attention layers. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Second, children become better word learners over the course of development60, similar to a meta-learner improving with training.”
§ Human-like systematic generalization through a meta-learning neural network > Discussion
What the sources say
- Supportedcites [60] Bergelson, E. The comprehension boost in early word learning: older infants are better learners. Child Dev. Perspect. 14, 142–149 (2020). source
The Abstract reports a robust improvement in infants’ word comprehension after their first birthday and proposes that older infants learn to make better use of relatively stable language input. The Language Input section likewise says children take greater advantage of that input. The meta-learner comparison is the citing authors’ analogy, not a claim attributed to this source. (LLM, conf=0.90)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“COGS is a multi-faceted benchmark that evaluates many forms of systematic generalization.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
What the sources say
- Supportedcites [16] Kim, N. & Linzen, T. COGS: a compositional generalization challenge based on semantic interpretation. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP) 9087–9105 (2020). source
The Overview of COGS (pp. 2–3) describes a challenge set targeting five categories of generalization, with a full list of cases in Table 1. The Introduction (p. 1) names examples including novel combinations of words and grammatical roles, deeper phrase nesting, and verb argument alternation. This supports describing COGS as a benchmark evaluating multiple forms of systematic generalization. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After considering seven different models, we found that, in contrast to perfectly systematic but rigid probabilistic symbolic models, and perfectly flexible but unsystematic neural networks, only MLC achieves both the systematicity and flexibility needed for human-like generalization.”
§ Human-like systematic generalization through a meta-learning neural network > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the few-shot instruction task, this improves the test loss marginally, but not accuracy.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Alternative neural and symbolic models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For successful optimization, it is also important to pass each study example (input sequence only) as an additional query when training on a particular episode.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Implementation of MLC
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“The current architecture also lacks a mechanism for emitting new symbols2, although new symbols introduced through the study examples could be emitted through an additional pointer mechanism55.”
§ Human-like systematic generalization through a meta-learning neural network > Discussion
What the sources say
- Not matchedcites [2] Marcus, G. F. The Algebraic Mind: Integrating Connectionism and Cognitive Science (MIT Press, 2003).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [55] Vinyals, O., Fortunato, M. & Jaitly, N. Pointer networks. In Proc. Advances in Neural Information Processing Systems (eds Cortes, C. et al.) (Curran Associates, 2015).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In our experiments, only MLC closely reproduced human behaviour with respect to both systematicity and biases, with the MLC (joint) model best navigating the trade-off between these two blueprints of human linguistic behaviour.”
§ Human-like systematic generalization through a meta-learning neural network > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“MLC also outperforms a basic seq2seq transformer fit to the patterns in Fig. 2 without meta-learning and an MLC model optimized for copying rather than systematic generalization (MLC (copy only); during training, the query examples always match one of the study examples).”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The algebraic interpretation grammar provides the correct normative standard for the few-shot instruction-learning task.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The MLC transformer responded exactly like the modal human participant in 65.0% of samples (Fig. 3b (left)), perfectly instantiating the three key inductive biases.”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The algebraic interpretation grammar provides the correct normative standard for the few-shot instruction-learning task.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Despite the unconstrained nature of the test, people’s responses were highly structured and confirm the previous two inductive biases.”
§ Human-like systematic generalization through a meta-learning neural network > Behavioural results
plaincorrelationaln = 29- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Human participants recruited via Amazon Mechanical Turk are representative of the broader human population for these compositional skills.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“MLC shows much stronger systematicity than neural networks trained in standard ways, and shows more nuanced behaviour than pristine symbolic models.”
§ Human-like systematic generalization through a meta-learning neural network > Discussion
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However surface-level permutations were not enough for MLC to solve the structural generalization tasks in the benchmarks. MLC fails to handle longer output sequences (SCAN length split) as well as novel and more complex sentence structures (three types in COGS), with error rates at 100%.”
§ Human-like systematic generalization through a meta-learning neural network > Machine learning benchmarks
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Chance performance is 2.8% for two-length output sequences if the length is known, and exponentially less for longer sequences.”
§ Human-like systematic generalization through a meta-learning neural network > Behavioural results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nevertheless, all 10 runs failed to generalize systematically on the few-shot instruction task (0% exact-match accuracy).”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Alternative neural and symbolic models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, we evaluated lower-capacity transformers but found that they did not perform better.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Alternative neural and symbolic models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On SCAN, MLC solves three systematic generalization splits with an error rate of 0.22% or lower (99.78% accuracy or above), including the already mentioned ‘add jump’ split and ‘around right’ and ‘opposite right’, which examine novel combinations of known words.”
§ Human-like systematic generalization through a meta-learning neural network > Machine learning benchmarks
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“However, people also relied on inductive biases that sometimes support the algebraic solution and sometimes deviate from it; indeed, people are not purely algebraic machines3,6,7.”
§ Human-like systematic generalization through a meta-learning neural network > Discussion
What the sources say
- Not checked — no full text of the sourcecites [3] Johnson, K. On the systematicity of language and thought. J. Philos. 101, 111–139 (2004). source
- Not checked — no full text of the sourcecites [6] O’Reilly, R. C. et al. in The Architecture of Cognition: Rethinking Fodor and Pylyshyn’s Systematicity Challenge (eds Calvo, P. & Symons, J.) 191–226 (MIT Press, 2014). source
- Not matchedcites [7] Nam, A. J. & McClelland, J. L. What underlies rapid learning and systematic generalization in humans? Preprint at http://arxiv.org/abs/2107.06994 (2021).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The algebraic interpretation grammar provides the correct normative standard for the few-shot instruction-learning task.
- Rests on an assumption: Human participants recruited via Amazon Mechanical Turk are representative of the broader human population for these compositional skills.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Indeed, MLC was similarly optimized to (implicitly) infer systematic rules and respond with the same biased-based patterns, and it is therefore natural that the two models would perform similarly.”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“MLC was evaluated on this task in several ways; in each case, MLC responded to this novel task through learned memory-based strategies, as its weights were frozen and not updated further.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Implementation of MLC
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“Model predictions were also evaluated through fivefold cross-validation33: MLC and other models were optimized on responses for either 23 or 24 participants (depending on the cross-validation split) and then predicted responses for held-out participants.”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
What the sources say
- Not matchedcites [33] Myung, J. I. & Pitt, M. A. in Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience (ed. Wixted, J. T.) 85–118 (John Wiley & Sons, 2018).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainn = 24- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The study items remained on the screen for reference, so that performance would reflect generalization in the absence of memory limitations.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Behavioural methods: few-shot learning task
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, MLC did handle novel sentence structures in our few-shot instruction-learning task (77.8% correct on queries with both longer input and output sequences than seen during study; Fig. 2), suggesting that the right meta-training procedure can promote productivity—a challenge we leave to future work.”
§ Human-like systematic generalization through a meta-learning neural network > Machine learning benchmarks
hedgedconditionalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Beyond natural language, people require a years-long process of education to master other forms of systematic generalization and symbolic reasoning6,7, including mathematics, logic and computer programming.”
§ Human-like systematic generalization through a meta-learning neural network > Discussion
What the sources say
- Not checked — no full text of the sourcecites [6] O’Reilly, R. C. et al. in The Architecture of Cognition: Rethinking Fodor and Pylyshyn’s Systematicity Challenge (eds Calvo, P. & Symons, J.) 191–226 (MIT Press, 2014). source
- Not matchedcites [7] Nam, A. J. & McClelland, J. L. What underlies rapid learning and systematic generalization in humans? Preprint at http://arxiv.org/abs/2107.06994 (2021).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The study and test items always differed from one another by more than one primitive substitution (except in the function 1 stage, where a single primitive was presented as a novel argument to function 1).”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Behavioural methods: few-shot learning task
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that here the examples used for optimization were generated by the benchmark designers through algebraic rules, and there is therefore no direct imitation of human behavioural data.”
§ Human-like systematic generalization through a meta-learning neural network > Machine learning benchmarks
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The study and test set only used four output symbols, but the pool provided six possibilities (that is, there were two extra colours that were not associated to words), to discourage reasoning by exclusion.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Behavioural methods: few-shot learning task
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The COGS output expressions were converted to uppercase to remove any incidental overlap between input and output token indices (which MLC, but not basic seq2seq, could exploit).”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This more scalable MLC variant, the original MLC architecture (see the ‘Architecture and optimizer’ section) and basic seq2seq all have approximately the same number of learnable parameters (except for the fact that basic seq2seq has a smaller input vocabulary).”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“MLC shows how meta-learning can be used like hierarchical Bayesian models for reverse-engineering inductive biases (see ref. 47 for a formal connection), although with the aid of neural networks for greater expressive power.”
§ Human-like systematic generalization through a meta-learning neural network > Discussion
What the sources say
- Not matchedcites ref. 47
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The decoder cross-attention layers are less expensive, O(ST), because the target sequence length T, which does not include any study examples, is typically much shorter (T ≪ S).”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Our experimental paradigm asks participants to process instructions in a pseudolanguage in order to generate abstract outputs (meanings), differing from artificial grammar learning22, statistical learning23 and program learning24 in that explicit or implicit judgments of grammaticality are not needed.”
§ Human-like systematic generalization through a meta-learning neural network > Behavioural results
What the sources say
- Not checked — no full text of the sourcecites [22] Reber, A. Implicit learning of artificial grammars. Verb. Learn. Verb. Behav. 5, 855–863 (1967). source
- Not checked — no full text of the sourcecites [23] Aslin, R. N., Saffran, J. R. & Newport, E. L. Computation of conditional probability statistics by 8-month-old infants. Psychol. Sci. 9, 321–324 (1998). source
- Not matchedcites [24] Stuhlmuller, A., Tenenbaum, J. B. & Goodman, N. D. Learning structured generative concepts. In Proc. Thirty-Second Annual Conference of the Cognitive Science Society, 2296–2301 (2010).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“using the test corpus for study examples would inadvertently leak test information”
- Also stated in: § Machine learning benchmarks · hedged
“picking study examples from the generalization corpus would inadvertently leak test information”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § Machine learning benchmarks · hedged
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Finally, query responses were also excluded if the corresponding study phases were not completed correctly (for all items) within three attempts (13% of remaining data).”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Behavioural methods: few-shot learning task
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The validation episodes were defined by new grammars that differ from the training grammars. Grammars were only considered new if they did not match any of the meta-training grammars, even under permutations of how the rules are ordered.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Implementation of MLC
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“First, before the experiment, the participants practiced using the response interface and had to pass an instructions quiz; they cycled through the quiz until they passed it.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Behavioural methods: few-shot learning task
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We used several strategies to ensure that our participants were paying attention.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Behavioural methods: few-shot learning task
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“For statistical analyses of the data from this experiment and elsewhere, we tested for data normalcy and applied alternative nonparametric or permutation tests when the assumptions were not met.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Behavioural methods: few-shot learning task
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All of the studies were approved by the NYU IRB, protocol FY2018-1728, and obtained informed consent.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Behavioural methods: few-shot learning task
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Importantly, this approach seeks to model adult compositional skills but not the process by which adults acquire those skills, which is an issue that is considered further in the general discussion.”
§ Human-like systematic generalization through a meta-learning neural network > Modelling results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The vocabulary in COGS is much larger than in SCAN; thus, the study examples cannot be sampled arbitrarily with any reasonable hope that they would inform the query of interest.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our implementation of MLC uses only common neural networks without added symbolic machinery, and without hand-designed internal representations or inductive biases.”
§ Human-like systematic generalization through a meta-learning neural network > Main
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Without access to study examples, the model is poorly equipped for learning words with changing meanings; it has no in-context memory and, therefore, all of its knowledge must be stored in the learned weights.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Alternative neural and symbolic models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, MLC derives its abilities through meta-learning, where both systematic generalization and the human biases are not inherent properties of the neural network architecture but, instead, are induced from data.”
§ Human-like systematic generalization through a meta-learning neural network > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, catch trials were included during the test phases, probing the study items rather than new items, with the answers clearly presented on the screen above. There was one catch trial per stage (except the last stage had two); participants were excluded if they missed two or more catch trials (n = 5).”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Behavioural methods: few-shot learning task
plainn = 5- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“One participant was excluded because they reported using an external aid in a post-test survey.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Behavioural methods: open-ended task
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Occasionally skipping the permutations in this way helps to break symmetries that can slow optimization; that is, the association between the input and output primitives is no longer perfectly balanced.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The gold interpretation grammar that produced the few-shot instruction-learning task with humans and machines (Extended Data Fig. 2) was also reserved for testing in this way, with an additional structural requirement that grammars for producing the training and validation episodes should also not match the gold grammar through any permutation of the input and output symbol assignments.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Implementation of MLC
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“for the ‘add jump’ split, one study example is fixed to be ‘jump → JUMP’, ensuring that MLC has access to the basic meaning before attempting compositional uses of ‘jump’.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Performance was averaged over 200 passes through the dataset, each episode with different random query orderings as well as word and colour assignments.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Implementation of MLC
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“For comparison with the gold grammar or with human behaviour via log-likelihood, performance was averaged over 100 random word/colour assignments.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Implementation of MLC
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The assignment of words to colours and functions was randomized for each participant (drawn from nine possible words and six colours), and the first three stages were presented in random order.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Behavioural methods: few-shot learning task
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To make sense of an episode, MLC must become adept at inferring, from just a few study examples, how words map to meanings.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To master the lexical generalization splits, the meta-training procedure targets several lexical classes that participate in particularly challenging compositional generalizations.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- The authors state this as an assumption: Broad lexical classes (common nouns, proper nouns, dative verbs, infinitive verbs) are assumed.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Note that several other input words (the mostly ‘functional’ words ‘turn’, ‘twice’, ‘thrice’, ‘around’, ‘opposite’, ‘and’, ‘after’) have stable meanings that can be stored in the model weights.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Importantly, although the broad classes are assumed and could plausibly arise through simple distributional learning68,69, the correspondence between input and output word types is unknown and not used.”
- Also stated in: § Machine learning benchmarks · plain
“As before, the correspondence between input and output tokens is unknown, both at the levels of a sentence and the whole dataset.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Also stated in: § Machine learning benchmarks · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“MLC must also become adept at composition: it must systematically compose the inferred word meanings to correctly answer the queries.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Other verbs, punctuation and logical symbols have stable meanings that can be stored in the model weights.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Machine learning benchmarks
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that all of the models compared in Table 1 have the same opportunity to fit a lapse parameter.”
§ Human-like systematic generalization through a meta-learning neural network > Methods > Alternative neural and symbolic models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
The algebraic interpretation grammar provides the correct normative standard for the few-shot instruction-learning task.
“The study instructions were consistent with an underlying interpretation grammar, which derives outputs from inputs through a set of compositional rewrite rules (see the ‘Interpretation grammars’ section of the Methods).”
The paper relies on this without support.
Human participants recruited via Amazon Mechanical Turk are representative of the broader human population for these compositional skills.
“All experiments were run on Amazon Mechanical Turk, and detailed procedures are described in the ‘Behavioural methods: few-shot learning task’ and ‘Behavioural methods: open-ended task’ sections of the Methods.”
The authors state this as an assumption.
Broad lexical classes (common nouns, proper nouns, dative verbs, infinitive verbs) are assumed.
“Importantly, although the broad classes are assumed and could plausibly arise through simple distributional learning68,69, the correspondence between input and output word types is unknown and not used.”
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212
Connectionism and cognitive architecture: A critical analysis
Fodor, J. A., & Pylyshyn, Z. W. (1988) · Cognition · doi:10.1016/0010-0277(88)90031-5 · Cited by 4,022
Gaussian Error Linear Units (GELUs)
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1606.08415 · Cited by 3,537
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511
Implicit learning of artificial grammars
Reber, A. S. (1967) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(67)80149-x · Cited by 2,026
Children's use of mutual exclusivity to constrain the meanings of words
Markman, E. M., & Wachtel, G. F. (1988) · Cognitive Psychology · doi:10.1016/0010-0285(88)90017-5 · Cited by 1,337
Computation of conditional probability statistics by 8-month-old infants
Aslin, R. N., Saffran, J. R., & Newport, E. L. (1998) · Psychological Science · doi:10.1111/1467-9280.00063 · Cited by 1,189
Smolensky, P. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90007-m · Cited by 959
Research works printed but not linked: 21
- Marcus, G. F. The Algebraic Mind: Integrating Connectionism and Cognitive Science (MIT Press, 2003).
- Hill, F. et al. Environmental drivers of systematicity and generalisation in a situated agent. In Proc. International Conference on Learning Representations (ICLR) (2020).
- Lake, B. M. & Baroni, M. Generalization without systematicity: on the compositional skills of sequence-to-sequence recurrent networks. In Proc. International Conference on Machine Learning (ICML) (eds. Dy, J. & Krause, A.) 2873–2882 (PMLR, 2018).
- Ettinger, A., Elgohary, A., Phillips, C. & Resnik, P. Assessing composition in sentence vector representations. In Proc. 7th International Conference on Computational Linguistics, (COLING 2018) 1790–1801 (Association for Computational Linguistics, 2018).
- Stuhlmuller, A., Tenenbaum, J. B. & Goodman, N. D. Learning structured generative concepts. In Proc. Thirty-Second Annual Conference of the Cognitive Science Society, 2296–2301 (2010).
- Sutskever, I., Vinyals, O. & Le, Q. V. Sequence to sequence learning with neural networks. In Proc. Advances in Neural Information Processing Systems (eds Ghahramani, Z. et al.) (Curran Associates, 2014).
- Lake, B. M. Compositional generalization through meta sequence-to-sequence learning. In Proc. Advances in Neural Information Processing Systems (NeurIPS) 32 (eds Wallach, H. et al.) 9791–9801 (Curran Associates, 2019).
- Myung, J. I. & Pitt, M. A. in Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience (ed. Wixted, J. T.) 85–118 (John Wiley & Sons, 2018).
- Chen, X., Liang, C., Yu, A. W., Song, D. & Zhou, D. Compositional generalization via neural-symbolic stack machines. In Proc. Advances in Neural Information Processing Systems 33 (eds Larochelle, H. et al.) 1690–1701 (Curran Associates, 2020).
- Russin, J., Jo, J., O’Reilly, R. C. & Bengio, Y. Systematicity in a recurrent neural network by factorizing syntax and semantics. In Proc. 42nd Annual Meeting of the Cognitive Science Society (eds Denison, S. et al.) (Cognitive Science Society. 2020).
- Nye, M. I., Solar-Lezama, A., Tenenbaum, J. B. & Lake, B. M. Learning compositional rules via neural program synthesis. In Proc. Advances in Neural Information Processing Systems (NeurIPS) 33 (eds Larochelle, H. et al.) (Curran Associates, 2020).
- Singh, G., Deng, F. & Ahn, S. Illiterate DALL-E learns to compose. In Proc. ICLR https://openreview.net/group?id=ICLR.cc/2022/Conference (2022).
- Gandhi, K. & Lake, B. M. Mutual exclusivity as a challenge for deep neural networks. In Proc. Advances in Neural Information Processing Systems (NeurIPS) 33 (eds Larochelle, H. et al.) 14182–14192 (Curran Associates, 2020).
- Grant, E., Finn, C., Levine, S., Darrell, T. & Griffiths, T. Recasting gradient-based meta-learning as hierarchical bayes. In Proc. International Conference on Learning Representations (ICLR) (2019).
- Grant, E., Peterson, J. C. & Griffiths, T. Learning deep taxonomic priors for concept learning from few positive examples. In Proc. Annual Meeting of the Cognitive Science Society (eds Goel, A. K. et al.) 1865–1870 (Cognitive Science Society, 2019).
- Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D. & Lillicrap, T. Meta-learning with memory-augmented neural networks. In Proc. International Conference on Machine Learning (ICML) 1842–1850 (PMLR, 2016).
- McCoy, R. T., Grant, E., Smolensky, P., Griffiths, T. L. & Linzen, T. Universal linguistic inductive biases via meta-learning. In Proc. 42nd Annual Conference of the Cognitive Science Society (eds Denison, S. et al.) (Cognitive Science Society, 2020).
- Vinyals, O., Fortunato, M. & Jaitly, N. Pointer networks. In Proc. Advances in Neural Information Processing Systems (eds Cortes, C. et al.) (Curran Associates, 2015).
- Radford, A., Narasimhan, K. R., Salimans, T. & Sutskever, I. Improving language understanding by generative pre-training. Preprint at https://openai.com/research/language-unsupervised (2018).
- Mitchell, E., Finn, C. & Manning, C. Challenges of acquiring compositional inductive biases via meta-learning. In Proc. AAAI Workshop on Meta-Learning and MetaDL Challenge 138–148 (2021).
- Lake, B. M., Linzen, T. & Baroni, M. Human few-shot learning of compositional instructions. In Proc. 41st Annual Conference of the Cognitive Science Society (eds Goel, A. K. et al.) 611–617 (Cognitive Science Society, 2019).
Cited by
Sorted by citations
Artificial intelligence and illusions of understanding in scientific research
Messeri, L., & Crockett, M. J. (2024) · Nature · doi:10.1038/s41586-024-07146-0 · Cited by 707
Towards Generalist Biomedical AI
Tu, T., Azizi, S., Driess, D., Schaekermann, M., Amin, M., Chang, P.-C., Carroll, A., Lau, C., Tanno, R., Ktena, I., et al. (2024) · NEJM AI · doi:10.1056/aioa2300138 · Cited by 483
Language is primarily a tool for communication rather than thought
Fedorenko, E., Piantadosi, S. T., & Gibson, E. A. F. (2024) · Nature · doi:10.1038/s41586-024-07522-w · Cited by 190
Theory Is All You Need: AI, Human Cognition, and Causal Reasoning
(2024) · doi:10.1287/stsc.2024.0189 · Cited by 151
Nakaura, T., Ito, R., Ueda, D., Nozaki, T., Fushimi, Y., Matsui, Y., Yanagawa, M., Yamada, A., Tsuboyama, T., Fujima, N., et al. (2024) · Japanese Journal of Radiology · doi:10.1007/s11604-024-01552-0 · Cited by 131
Building machines that learn and think with people
Collins, K. M., Sucholutsky, I., Bhatt, U., Chandra, K., Wong, L., Lee, M., Zhang, C. E., Zhi‐Xuan, T., Ho, M. K., Mansinghka, V. K., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01991-9 · Cited by 125
Material Engineering Strategies for Efficient Hydrogen Evolution Reaction Catalysts
Luo, Y., Zhang, Y., Zhu, J., Tian, X., Liu, G., Feng, Z., Pan, L., Liu, X., Han, N., & Tan, R. Q. (2024) · Small Methods · doi:10.1002/smtd.202400158 · Cited by 100
A Survey of Autonomous Driving from a Deep Learning Perspective
Zhao, J., Wu, Y., Deng, R., Xu, S., Gao, J., & Burke, A. F. (2025) · ACM Computing Surveys · doi:10.1145/3729420 · Cited by 99
Global data-driven prediction of fire activity
Giuseppe, F. D., McNorton, J., Lombardi, A., & Wetterhall, F. (2025) · Nature Communications · doi:10.1038/s41467-025-58097-7 · Cited by 96
Hagendorff, T., Dasgupta, I., Binz, M., Chan, S. C. Y., Lampinen, A., Wang, J. X., Akata, Z., & Schulz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13988 · Cited by 68
Zhao, J., Han, X., Wu, Y., Wang, Z., & Burke, A. F. (2024) · Journal of Energy Chemistry · doi:10.1016/j.jechem.2024.11.011 · Cited by 66
Building compositional tasks with shared neural subspaces
Tafazoli, S., Bouchacourt, F., Ardalan, A., Markov, N. T., Uchimura, M., Mattar, M. G., Daw, N. D., & Buschman, T. J. (2025) · Nature · doi:10.1038/s41586-025-09805-2 · Cited by 58
Robot learning in the era of foundation models: a survey
Xiao, X., Liu, J., Wang, Z., Zhou, Y., Qi, Y., Jiang, S., He, B., & Cheng, Q. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129963 · Cited by 56
Modeling rapid language learning by distilling Bayesian priors into artificial neural networks
McCoy, R. T., & Griffiths, T. L. (2025) · Nature Communications · doi:10.1038/s41467-025-59957-y · Cited by 50
Why concepts are (probably) vectors
Piantadosi, S. T., Muller, D. C. Y., Rule, J., Kaushik, K., Gorenstein, M. I., Leib, E. R., & Sanford, E. M. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.06.011 · Cited by 51
Cheng, Y., Tian, Z., Ning, D., Feng, K., Li, Z., Chauhan, S., & Vashishtha, G. (2024) · Measurement · doi:10.1016/j.measurement.2024.116426 · Cited by 49
Discovery Learning predicts battery cycle life from minimal experiments
Zhang, J., Zhang, Y., Yi, B., Ren, Y., Jiao, Q., Bai, H., Jiang, W., & Song, Z. (2026) · Nature · doi:10.1038/s41586-025-09951-7 · Cited by 45
Language Models as Models of Language
Millière, R. (2026) · Oxford University Press eBooks · doi:10.1093/oso/9780198879640.003.0025 · Cited by 43
Accelerated optimization in deep learning with a proportional-integral-derivative controller
Chen, S., Liu, J., Wang, P., Xu, C., Cai, S., & Chu, J. (2024) · Nature Communications · doi:10.1038/s41467-024-54451-3 · Cited by 42
Data-driven shear capacity analysis of headed stud in steel-UHPC composite structures
Zhou, C., Wang, W., & Zheng, Y. (2024) · Engineering Structures · doi:10.1016/j.engstruct.2024.118946 · Cited by 39
Meta-learned models of cognition
Binz, M., Dasgupta, I., Jagadish, A. K., Botvinick, M., Wang, J. X., & Schulz, E. (2024) · Behavioral and Brain Sciences · doi:10.1017/s0140525x23003266 · Cited by 38
Aligning generalization between humans and machines
Ilievski, F., Hammer, B., van Harmelen, F., Paaßen, B., Saralajew, S., Schmid, U., Biehl, M. L., Bolognesi, M., Dong, X., Gashteovski, K., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01109-4 · Cited by 37
Ma, P., Chen, L., Yu, C., Zhu, Q., Ding, Y., Wu, Z., Li, H., Tian, C., & Fan, X. (2025) · Remote Sensing of Environment · doi:10.1016/j.rse.2025.114800 · Cited by 36
Cognitive flexibility as the shifting of brain network flows by flexible neural representations
Cole, M. W. (2024) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2024.101384 · Cited by 33
Appraising Regulatory Framework Towards Artificial General Intelligence (AGI) Under Digital Humanism
Cheng, L., & Gong, X. (2024) · International Journal of Digital Law and Governance · doi:10.1515/ijdlg-2024-0015 · Cited by 28
Towards Generalist Biomedical AI
Tu, T., Azizi, S., Driess, D., Schaekermann, M., Amin, M., Chang, P.-C., Carroll, A., Lau, C., Tanno, R., Ktena, I., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.14334 · Cited by 483
HyCubE: Efficient Knowledge Hypergraph 3D Circular Convolutional Embedding
Li, Z., Wang, X., Zhao, J., Guo, W., & Li, J. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3531372 · Cited by 27
Russin, J., Pavlick, E., & Frank, M. J. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2510270122 · Cited by 27
How Can Deep Neural Networks Inform Theory in Psychological Science?
McGrath, S. W., Russin, J., Pavlick, E., & Feiman, R. (2024) · Current Directions in Psychological Science · doi:10.1177/09637214241268098 · Cited by 26
Leivada, E., Marcus, G., Günther, F., & Murphy, E. (2026) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2025.0008 · Cited by 28
Reasoning in Computer Vision: Taxonomy, Models, Tasks, and MethodologiesPreprint
Sarkar, A., Yu, Z., & Idris, M. Y. I. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.10523 · Cited by 25
Deep neural networks and humans both benefit from compositional language structure
Galke, L. P. A., Ram, Y., & Raviv, L. (2024) · Nature Communications · doi:10.1038/s41467-024-55158-1 · Cited by 24
Revealing emergent human-like conceptual representations from language prediction
Xu, N., Zhang, Q., Du, C., Luo, Q., Qiu, X., Huang, X., & Zhang, M. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2512514122 · Cited by 21
CLIP-guided continual novel class discovery
Yan, Q., Yang, Y., Dai, Y., Zhang, X., Wiltos, K., Woźniak, M., Dong, W., & Zhang, Y. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112920 · Cited by 20
Delay-dependent bifurcation conditions in a fractional-order inertial BAM neural network
Huang, C., Wang, H., Cao, J., & Liu, H. (2024) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2024.115106 · Cited by 19
A mathematical theory of relational generalization in transitive inference
Lippl, S., Kay, K., Jensen, G., Ferrera, V. P., & Abbott, L. F. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314511121 · Cited by 20
Planning in a recurrent neural network that plays SokobanPreprint
Taufeeque, M., Quirke, P., Li, M. X., Cundy, C., Tucker, A. D., Gleave, A., & Garriga-Alonso, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.15421 · Cited by 19
Philosophy of cognitive science in the age of deep learning
Millière, R. (2024) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1684 · Cited by 18
Accelerating human–computer interaction through convergent conditions for LLM explanation
Raikov, A., Giretti, A., Pirani, M., Spalazzi, L., & Guo, M. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1406773 · Cited by 15
Neural representation of action symbols in primate frontal cortex
Tian, L. Y., Gupta, K. G., Hanuska, D. J., Rouse, A. G., Eldridge, M. A. G., Schieber, M. H., Wang, X., Tenenbaum, J. B., & Freiwald, W. A. (2026) · Nature · doi:10.1038/s41586-026-10297-x · Cited by 19
Attention as a HypernetworkPreprint
Schug, S., Kobayashi, S., Akram, Y., Sacramento, J., & Pascanu, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.05816 · Cited by 15
Mixtures Recomposition by Neural Nets: A Multidisciplinary Overview
Nicolle, A., Deng, S., Ihme, M., Kuzhagaliyeva, N., Ibrahim, E. A., & Farooq, A. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01633 · Cited by 14
General pre-trained inertial signal feature extraction based on temporal memory fusion
Wang, Y., & Zhao, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103274 · Cited by 14
Psychomatics—A Multidisciplinary Framework for Understanding Artificial Minds
Riva, G., Mantovani, F., Wiederhold, B. K., Marchetti, A., & Gaggioli, A. (2024) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2024.0409 · Cited by 14
Kim, J. H., & Kim, H. R. (2024) · Research Square · doi:10.21203/rs.3.rs-4328966/v1 · Cited by 14
Li, J., Pan, N., Yao, Y., Li, G., Cheng, Z., Lu, Y., Liu, S., & Liu, W. (2024) · Ecological Informatics · doi:10.1016/j.ecoinf.2024.102686 · Cited by 13
The Roles of Neural Networks in Language Acquisition
Portelance, E., & Jasbi, M. (2024) · Language and Linguistics Compass · doi:10.1111/lnc3.70001 · Cited by 13
Piray, P., & Daw, N. D. (2025) · Nature Communications · doi:10.1038/s41467-025-62733-7 · Cited by 16
Yager, K. G. (2024) · Digital Discovery · doi:10.1039/d4dd00178h · Cited by 12
Feyijimi, T. R., Aliu, J., Oke, A. E., & Aghimien, D. O. (2025) · Computers · doi:10.3390/computers14090366 · Cited by 12
Liao, Y., Tang, H., Li, R., Ran, L., & Xie, L. (2024) · Engineering Structures · doi:10.1016/j.engstruct.2024.118077 · Cited by 12
Systematic Human Learning and Generalization From a Brief Tutorial With Explanatory Feedback
Nam, A., & McClelland, J. L. (2024) · Open Mind · doi:10.1162/opmi_a_00123 · Cited by 12
Semi-supervised meta-learning elucidates understudied molecular interactions
Wu, Y., Xie, L., Liu, Y., & Xie, L. (2024) · Communications Biology · doi:10.1038/s42003-024-06797-z · Cited by 11
Kieser, F., Tschisgale, P., Rauh, S., Bai, X., Maus, H., Petersen, S., Stede, M., Neumann, K., & Wulff, P. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1408817 · Cited by 11
Whither symbols in the era of advanced neural networks?
Griffiths, T. L., Lake, B. M., McCoy, R. T., Pavlick, E., & Webb, T. W. (2026) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2026.02.003 · Cited by 14
Self-citationMa, W., & Guo, B. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e32679 · Cited by 11
Educational Concepts and Methodologies in the AI Era: Challenges and Responses
Dong, C. (2024) · Frontiers of digital education. · doi:10.1007/s44366-024-0022-y · Cited by 11
Position: Understanding LLMs Requires More Than Statistical GeneralizationPreprint
Reizinger, P., Ujváry, S., Mészáros, A., Kerekes, A., Brendel, W., & Huszár, F. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.01964 · Cited by 11
Introduction to Artificial Consciousness: History, Current Trends and Ethical ChallengesPreprint
Elamrani, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.05823 · Cited by 11
NARRATE: Versatile Language Architecture for Optimal Control in Robotics
Ismail, S., Arbues, A., Cotterell, R., Zurbrügg, R., & Alonso, C. A. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801425 · Cited by 10
MAML-XL: a symbolic music generation method based on meta-learning and Transformer-XL
Liang, Y., Abudukelimu, H., Chen, J., Abulizi, A., & Guo, W. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01803-8 · Cited by 10
A Survey of Human Intelligence Augmented Artificial Intelligence: An Autonomous Driving Perspective
Li, S., Yuan, K., Tang, Z., Zhang, Y., Ji, P., Ge, Q., Du, S., Huang, Y., & Chen, H. (2025) · Automotive Innovation · doi:10.1007/s42154-025-00361-z · Cited by 10
Automated discovery of algorithms from data
Blazek, P. J., Venkatesh, K., & Lin, M. M. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00593-9 · Cited by 10
Jeong, Y., Song, J., Yang, J., & Kang, S. M. (2024) · IEEE Access · doi:10.1109/access.2024.3383020 · Cited by 10
Overcoming classic challenges for artificial neural networks by providing incentives and practice
Irie, K., & Lake, B. M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01121-8 · Cited by 10
Self-citationThe Influence of AI in the Media Work Force: How Companies Use an Array of Legal Remedies
Díaz-Noci, J., Peña-Fernández, S., Ayerdi, K. M., & Ureta, A. L. (2024) · Tripodos · doi:10.51698/tripodos.2024.55.03 · Cited by 10
Quantifying artificial intelligence through algorithmic generalization
Ito, T., Campbell, M. S., Lior Horesh, C., Klinger, T., & Ram, P. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01092-w · Cited by 10
Bertin, D., Bongrand, P., & Bardin, N. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25063270 · Cited by 10
When does compositional structure yield compositional generalization? A kernel theoryPreprint
Lippl, S., & Stachenfeld, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.16391 · Cited by 9
Huang, H., Chen, M., Zhang, Y., Wang, X., Xie, Q., Pu, Y., He, Y., Zhu, L., He, Y., & Guo, J. (2025) · Langmuir · doi:10.1021/acs.langmuir.4c04262 · Cited by 8
Wang, Y., Liu, S., Lv, S., & Liu, G. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.13797 · Cited by 8
Natural Language Processing and Large Language Models
Wulff, P., Kubsch, M., & Krist, C. (2025) · Springer texts in education · doi:10.1007/978-3-031-74227-9_7 · Cited by 7
When can transformers compositionally generalize in-context?Preprint
Kobayashi, S., Schug, S., Akram, Y., Redhardt, F., von Oswald, J., Pascanu, R., Lajoie, G., & Sacramento, J. (2024) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-c-000794190 · Cited by 7
Standards for scale development in marketing: Elevating the role of theory
Hulland, J. S., Sample, K. L., & Houston, M. B. (2026) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-026-01153-5 · Cited by 6
Wang, W., Xiao, H., & Fang, Y. (2024) · International Journal of Clothing Science and Technology · doi:10.1108/ijcst-09-2023-0129 · Cited by 6
Reinforcement learning and meta-decision-making
Verbeke, P., & Verguts, T. (2024) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2024.101374 · Cited by 6
Promoting smartphone-based keratitis screening using meta-learning: A multicenter study
Li, Z., Wang, Y., Chen, K., Qiang, W., Zong, X., Ding, K., Wang, S., Yin, S., Jiang, J., & Chen, W. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104722 · Cited by 6
Liao, Y., Tang, H., & Xie, L. (2025) · Ocean Engineering · doi:10.1016/j.oceaneng.2025.120871 · Cited by 6
Computational basis of hierarchical and counterfactual information processing
Ramadan, M., Tang, C., Watters, N., & Jazayeri, M. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02232-3 · Cited by 7
AI ‘breakthrough’: neural net has human-like ability to generalize language
Kozlov, M., & Biever, C. (2023) · Nature · doi:10.1038/d41586-023-03272-3 · Cited by 6
Relational reasoning and inductive bias in transformers and large language modelsPreprint
Geerts, J. P., Liu, A., Chan, S. C. Y., Clopath, C., & Stachenfeld, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.04289 · Cited by 6
Valentine Joseph Owan, P. D. (2026) · Journal of Education, Innovation, and Communication. · doi:10.34097/jeicom-8-2-5 · Cited by 5
Meta-learning: Data, architecture, and both
Binz, M., Dasgupta, I., Jagadish, A. K., Botvinick, M., Wang, J. X., & Schulz, E. (2024) · Behavioral and Brain Sciences · doi:10.1017/s0140525x24000311 · Cited by 5
Adaptive curiosity about metacognitive ability.
Recht, S., Li, C., Yang, Y., & Chiu, K. (2024) · Journal of Experimental Psychology General · doi:10.1037/xge0001690 · Cited by 5
DeepPFP: a multi-task-aware architecture for protein function prediction
Wang, H., Ren, Z., Sun, J., Chen, Y., Bo, X., Xue, J., Gao, J., & Ni, M. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae579 · Cited by 5
Khadse, S., Gourshettiwar, P., & Pawar, A. (2025) · International Conference on Electronics and Renewable Systems (ICEARS) · doi:10.1109/icears64219.2025.10941123 · Cited by 5
The detection of algebraic auditory structures emerges with self-supervised learning
Orhan, P., Boubenec, Y., & King, J.-R. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013271 · Cited by 5
LLMs model how humans induce logically structured rules
Loo, A. M., Pavlick, E., & Feiman, R. (2025) · Journal of Memory and Language · doi:10.1016/j.jml.2025.104675 · Cited by 5
Evidence for compositionality in fMRI visual representations via Brain Algebra
Ferrante, M., Boccato, T., Toschi, N., & VanRullen, R. (2025) · Communications Biology · doi:10.1038/s42003-025-08706-4 · Cited by 4
Orbitofrontal-sensory cortical interactions in learning and adaptive decision-making
Rao, R., Six, H., Cortese, A., & Banerjee, A. (2025) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2025.10.017 · Cited by 4
Beyond sparsity: an empirical study of structured collaboration in modular AI
Zhou, X., Kim, S., Wang, Y., & Zhang, K. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131616 · Cited by 4
Shared sensitivity to data distribution during learning in humans and transformer networks
Lerousseau, J. P., & Summerfield, C. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02359-3 · Cited by 4
Human-like cognitive generalization for large models via mental representation-guided supervision
Chen, J., Qi, Y., Wang, Y., & Pan, G. (2026) · Nature Communications · doi:10.1038/s41467-026-71267-5 · Cited by 5
A Game-Theoretic Analysis of AI Integration in Education
Zhao, J., & Ding, R. (2024) · International Conference on Intelligent Education and Intelligent Research (IEIR) · doi:10.1109/ieir62538.2024.10960262 · Cited by 4
Scaling can lead to compositional generalization
Redhardt, F., Akram, Y., & Schug, S. (2025) · neural information processing systems · doi:10.52202/085713-1858 · Cited by 4
Is human compositionality meta-learned?
Russin, J., McGrath, S. W., Pavlick, E., & Frank, M. J. (2024) · Behavioral and Brain Sciences · doi:10.1017/s0140525x24000189 · Cited by 4
Exploring Continual Learning of Compositional Generalization in NLI
Fu, X., & Frank, A. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00680 · Cited by 4
Multiple Realizability and the Rise of Deep LearningPreprint
McGrath, S. W., & Russin, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.13231 · Cited by 4
Should Artificial Intelligence Play a Durable Role in Biomedical Research and Practice?
Bongrand, P. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms252413371 · Cited by 3
Developing artificial mechanics intuitions from extremely small data
Peng, J., & Zhu, S. (2025) · Extreme Mechanics Letters · doi:10.1016/j.eml.2025.102371 · Cited by 3
Metadata
- Authors
- Lake, B. M., & Baroni, M.
- Affiliations
- New York University (US)
- Institució Catalana de Recerca i Estudis Avançats (ES)
- Corresponding author
- Lake, Brenden M.
- Venue
- Nature
- Published
- 2023-10-25
- Volume
- 623
- Issue
- 7985
- Pages
- 115-121
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC10620072
- OpenAlex
- W4387949708
- Semantic Scholar
- dd4dfee7ad7a2ed179f9b2e80b83685b37661dbf
- Cited by
- 230
- References
- 48
- Citations by year
- 2026: 66 · 2025: 87 · 2024: 71 · 2023: 6
- FWCI
- 16.38
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Nature (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Repositori digital de la UPF (Universitat Pompeu Fabra) (submittedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- systematic generalization, compositional generalization, meta-learning, human-machine comparison
- MeSH terms
- Machine Learning, Humans, Language, Verbal Behavior, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
