Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning
Open accessAbstract
Human reasoning can often be understood as an interplay between two systems: the intuitive and associative ("System 1") and the deliberative and logical ("System 2"). Neural sequence models -- which have been increasingly successful at performing complex, structured tasks -- exhibit the advantages and failure modes of System 1: they are fast and learn patterns from data, but are often inconsistent and incoherent. In this work, we seek a lightweight, training-free means of improving existing System 1-like sequence models by adding System 2-inspired logical reasoning. We explore several variations on this theme in which candidate generations from a neural sequence model are examined for logical consistency by a symbolic reasoning module, which can either accept or reject the generations. Our approach uses neural inference to mediate between the neural System 1 and the logical System 2. Results in robust story generation and grounded instruction-following show that this approach can increase the coherence and accuracy of neurally-based generations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
GPT-3 responded with the intuitive but incorrect answer on all three items of the Cognitive Reflection Test.
“For all three items of the CRT, GPT-3 responded with the intuitive but incorrect answer, as has been observed in earlier work (15).”
From Using cognitive psychology to understand GPT-3 (2023) · cites this paper as [15]
The Introduction (p. 2) shows GPT-3 giving the intuitive but incorrect answer to the bat-and-ball CRT item. Appendix A (p. 14) discusses additional CRT questions but says GPT-3 makes the same mistakes as humans “often (but not always)”; the text does not establish that it gave that answer on all three original items. (LLM, conf=0.83)
Experimental — accuracy not yet measuredNye et al. (2021) improve the performance of LLMs in story generation and instruction-following tasks by using a symbolic reasoning module to check the logical consistency of generated text against a minimal world model.
“Similarly, Nye et al. (2021) improve the performance of LLMs in story generation and instruction-following tasks by using a symbolic reasoning module to check the logical consistency of generated text against a minimal world model.”
From Reliable Reasoning Beyond Natural Language (2024) · cites this paper as Nye et al., 2021
The Introduction (p. 2) and story-generation results (p. 5) support the claim that a symbolic world model checks GPT-3-generated stories for logical consistency and improves their coherence. But grounded instruction following uses an LSTM to generate action sequences, checking them against predicted goal locations—not checking generated text from an LLM against a minimal world model (Section 5, p. 8). (LLM, conf=0.89)
Experimental — accuracy not yet measuredLLMs are better suited for data extraction than for complex reasoning, particularly in scenarios requiring grounded statements and logical inference.
“According to (Nye et al. 2021;Ren et al. 2025), LLMs are better suited for data extraction than for complex reasoning, particularly in scenarios requiring grounded statements and logical inference.”
From Integrating Answer Set Programming and Large Language Models for Reliable Structured Knowledge Extraction (2026) · cites this paper as Nye et al. 2021
Section 3 (p. 4) uses a neural model to extract facts while a separate symbolic module checks logical constraints. The results (p. 5) report that GPT-3 paired with fact extraction and a world model outperformed GPT-3 alone on a reasoning task. This supports the distinction, but the paper does not establish that LLMs generally are better suited to data extraction than to complex reasoning across settings. (LLM, conf=0.84)
Experimental — accuracy not yet measuredOur approach builds on and extends LINC and Nye et al. (2021).
“Our approach builds on and extends the work of LINC (Olausson et al., 2023) and Nye et al. (2021).”
From Reliable Reasoning Beyond Natural Language (2024) · cites this paper as Nye et al., 2021
Several studies have explored integrating LLMs with symbolic reasoning modules.
“Several studies have explored integrating LLMs with symbolic reasoning modules (Olausson et al., 2023;Nye et al., 2021).”
From Reliable Reasoning Beyond Natural Language (2024) · cites this paper as Nye et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522 · see citing works
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works
de Moura, L., & Bjørner, N. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-78800-3_24 · Cited by 6,595 · see citing works
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256 · see citing works
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550 · see citing works
Self-citationZero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417 · see citing works
In two minds: dual-process accounts of reasoning
Evans, J. S. B. T. (2003) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.08.012 · Cited by 2,252 · see citing works
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merriënboer, B., Joulin, A., & Mikolov, T. (2015) · arXiv · doi:10.48550/arxiv.1502.05698 · Cited by 670 · see citing works
Mao, J., Gan, C., Kohli, P., Tenenbaum, J. B., & Wu, J. (2019) · arXiv · doi:10.48550/arxiv.1904.12584 · Cited by 444 · see citing works
Self-citationNeural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision
Chen, L., Berant, J., Le, Q. V., Forbus, K. D., & Lao, N. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1003 · Cited by 387 · see citing works
Inductive biases for deep learning of higher-level cognition
Goyal, A., & Bengio, Y. (2022) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2021.0068 · Cited by 300 · see citing works
Dialogue Natural Language Inference
Welleck, S., Weston, J., Szlam, A., & Cho, K. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1363 · Cited by 249 · see citing works
Learning to Write with Cooperative Discriminators
Holtzman, A., Buys, J., Forbes, M., Bosselut, A., Golub, D., & Choi, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1152 · Cited by 192 · see citing works
Constrained Language Models Yield Few-Shot Semantic Parsers
Shin, R., Lin, C., Thomson, S., Chen, C., Roy, S., Platanios, E. A., Pauls, A., Klein, D., Eisner, J., & Van Durme, B. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.608 · Cited by 139 · cited 1× · see citing works
“We found that this simple parsing technique works well and could easily be applied to other parsing-based tasks, as in Shin et al. (2021).”
MethodDreamCoder: growing generalizable, interpretable knowledge with wake–sleep Bayesian program learning
Ellis, K., Wong, L., Nye, M., Sablé-Meyer, M., Cary, L., Anaya Pozo, L., Hewitt, L., Solar-Lezama, A., & Tenenbaum, J. B. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0050 · Cited by 125 · see citing works
Self-citationLearning executable semantic parsers for natural language understanding
Liang, P. (2016) · Communications of the ACM · doi:10.1145/2866568 · Cited by 115 · see citing works
CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text
Sinha, K., Sodhani, S., Dong, J., Pineau, J., & Hamilton, W. L. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1458 · Cited by 110 · see citing works
Task-Oriented Dialogue as Dataflow Synthesis
Andreas, J., Bufe, J., Burkett, D., Chen, C., Clausman, J., Crawford, J., Crim, K., DeLoach, J., Dorner, L., Eisner, J., et al. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00333 · Cited by 98 · see citing works
A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generation
Xu, J., Ren, X., Zhang, Y., Zeng, Q., Cai, X., & Sun, X. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1462 · Cited by 87 · see citing works
Event Representations for Automated Story Generation with Deep Neural Nets
Martin, L. J., Ammanabrolu, P., Wang, X., Hancock, W. S., Singh, S., Harrison, B. E., & Riedl, M. O. (2017) · Maryland Shared Open Access Repository (USMAI Consortium) · doi:10.13016/m2w5sl-34p0 · Cited by 86 · see citing works
Implicit Representations of Meaning in Neural Language Models
Li, B. Z., Nye, M., & Andreas, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.143 · Cited by 84 · see citing works
Self-citationA speech understanding system based on statistical representation of semantics
Pieraccini, R., Tzoukermann, E., Gorelov, Z., Gauvain, J.-L., Levin, E., Lee, C.-H., & Wilpon, J. G. (1992) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1992.225939 · Cited by 84 · see citing works
Cross-domain Semantic Parsing via Paraphrasing
Su, Y., & Yan, X. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1127 · Cited by 79 · see citing works
NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints
Lu, X., West, P., Zellers, R., Bras, R. J. L., Bhagavatula, C., & Choi, Y. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.339 · Cited by 79 · see citing works
Span-based Semantic Parsing for Compositional Generalization
Herzig, J., & Berant, J. (2021) · doi:10.18653/v1/2021.acl-long.74 · Cited by 74 · see citing works
A Benchmark for Systematic Generalization in Grounded Language UnderstandingPreprint
Ruis, L., Andreas, J., Baroni, M., Bouchacourt, D., & Lake, B. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.05161 · Cited by 66 · see citing works
Self-citationLearning Compositional Rules via Neural Program SynthesisPreprint
Nye, M., Solar-Lezama, A., Tenenbaum, J. B., & Lake, B. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.05562 · Cited by 60 · see citing works
Self-citationPragmatically Informative Text Generation
Shen, S., Fried, D., Andreas, J., & Klein, D. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1410 · Cited by 53 · see citing works
PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation
Hua, X., & Wang, L. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.57 · Cited by 45 · see citing works
Controlling Style in Generated DialoguePreprint
Smith, E. M., Gonzalez-Rico, D., Dinan, E., & Boureau, Y.-L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.10855 · Cited by 38 · see citing works
Unnatural Language Processing: Bridging the Gap Between Synthetic and Natural Language DataPreprint
Marzoev, A., Madden, S. R., Kaashoek, M. F., Cafarella, M., & Andreas, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.13645 · Cited by 25 · see citing works
Generative Language-Grounded Policy in Vision-and-Language Navigation with Bayes' RulePreprint
Kurita, S., & Cho, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.07783 · Cited by 7 · see citing works
Think before you act: A simple baseline for compositional generalizationPreprint
Heinze‐Deml, C., & Bouchacourt, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.13962 · Cited by 7 · see citing works
Value-based Search in Execution Space for Mapping Instructions to Programs
Muhlgay, D., Herzig, J., & Berant, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1193 · Cited by 6 · see citing works
Flagged first, then by citations
3 of the 39 references couldn’t be matched to a known paper.
Research works printed but not linked: 15
- Deng, Y., Bakhtin, A., Ott, M., Szlam, A., and Marc'Aurelio Ranzato. Residual energy-based models of text generation. In International Conference on Learning Representations (ICLR) , pp. 1-18, 2020.
- Devlin, J., Uesato, J., Bhupatiraju, S., Singh, R., Mohamed, A.-r., and Kohli, P. Robustfill: Neural program learning under noisy i/o. ICML , 2017.
- Garcez, A. d. and Lamb, L. C. Neurosymbolic ai: The 3rd wave. arXiv preprint arXiv:2012.05876 , 2020. · arXiv
- Goyal, A., Didolkar, A., Ke, N. R., Blundell, C., Beaudoin, P., Heess, N., Mozer, M., and Bengio, Y. Neural production systems. CoRR , abs/2103.01937, 2021. URL https://arxiv.org/abs/ 2103.01937 . · arXiv
- Lake, B. M. and Murphy, G. L. Word meaning in minds and machines. arXiv preprint arXiv:2008.01766 , 2020. · arXiv
- Miao, N., Zhou, H., Mou, L., Yan, R., and Li, L. Cgmh: Constrained sentence generation by metropolis-hastings sampling. In AAAI , 2019.
- Mou, L., Lu, Z., Li, H., and Jin, Z. Coupling distributed and symbolic execution for natural language queries. In Precup, D. and Teh, Y. W. (eds.), Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning Research , pp. 2518-2526. PMLR, 06-11 Aug 2017. URL http://proceedings.mlr.press/v70/mou17a.html .
- Nie, Y., Williamson, M., Bansal, M., Kiela, D., and Weston, J. I like fish, especially dolphins: Addressing contradictions in dialogue modelling. arXiv preprint arXiv:2012.13391 , 2020. · arXiv
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9, 2019.
- Schlag, I. and Schmidhuber, J. Learning to reason with third-order tensor products. CoRR , abs/1811.12143, 2018. URL http://arxiv.org/abs/1811.12143 . · arXiv
- Serafini, L. and d'Avila Garcez, A. S. Logic tensor networks: Deep learning and logical reasoning from data and knowledge. CoRR , abs/1606.04422, 2016. URL http://arxiv.org/abs/1606. 04422 . · arXiv
- Spelke, E. S. and Kinzler, K. D. Core knowledge. Developmental Science , 10(1):89-96, 2007.
- Tsamoura, E., Hospedales, T., and Michael, L. Neural-symbolic integration: A compositional perspective. Proceedings of the AAAI Conference on Artificial Intelligence , 35(6):5051-5060, May 2021. URL https://ojs.aaai.org/index.php/AAAI/article/view/16639 .
- Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., and Dinan, E. Recipes for safety in open-domain chatbots. arXiv preprint arXiv:2010.07079 , 2020a. · arXiv
- Xu, S., Semnani, S. J., Campagna, G., and Lam, M. S. Autoqa: From databases to qa semantic parsers with only synthetic training data. arXiv preprint arXiv:2010.04806 , 2020b. · arXiv
Cited by
Sorted by citations
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829 · see citing works
“Nye et al. (2021) improve the logical consistency of samples from a System 1 model by adding a System 2-inspired logical reasoning module.”
MethodUsing cognitive psychology to understand GPT-3
Binz, M., & Schulz, E. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2218523120 · Cited by 559 · see citing works
“For all three items of the CRT, GPT-3 responded with the intuitive but incorrect answer, as has been observed in earlier work 22 .”
ResultMaieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
Jung, J., Qin, L., Welleck, S., Brahman, F., Bhagavatula, C., Bras, R. J. L., & Choi, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.82 · Cited by 117 · see citing works
Who is GPT-3? An exploration of personality, values and demographics
Miotto, M., Rossberg, N., & Kleinberg, B. (2022) · Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) · doi:10.18653/v1/2022.nlpcss-1.24 · Cited by 103 · see citing works
A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing
Gómez‐Rodríguez, C., & Williams, P. A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.966 · Cited by 81 · see citing works
Language models, like humans, show content effects on reasoning tasks
Lampinen, A. K., Dasgupta, I., Chan, S. C. Y., Sheahan, H. R., Creswell, A., Kumaran, D., McClelland, J. L., & Hill, F. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae233 · Cited by 69 · see citing works
Neuro-Symbolic Explainable Artificial Intelligence Twin for Zero-Touch IoE in Wireless Network
Munir, M. S., Kim, K. T., Adhikary, A., Saad, W., Shetty, S. S., Park, S.-B., & Hong, C. S. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3303713 · Cited by 47 · see citing works
Solving Math Word Problems via Cooperative Reasoning induced Language Models
Zhu, X., Wang, J., Zhang, L., Zhang, Y., Huang, Y., Gan, R., Zhang, J., & Yang, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.245 · Cited by 32 · see citing works
Minding Language Models’ (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker
Sclar, M., Kumar, S., West, P., Suhr, A., Choi, Y., & Tsvetkov, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.780 · Cited by 27 · see citing works
SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting
Zhang, X., Peng, B., Li, K., Zhou, J., & Meng, H. M. L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.891 · Cited by 27 · see citing works
Leveraging Large Language Models to Generate Answer Set Programs
Ishay, A., Yang, Z., & Lee, J. (2023) · Proceedings of the Twentieth International Conference on Principles of Knowledge Representation and Reasoning · doi:10.24963/kr.2023/37 · Cited by 25 · see citing works
Relational memory-augmented language models
Liu, Q., Yogatama, D., & Blunsom, P. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00476 · Cited by 23 · see citing works
Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming
Zhang, H., Huang, J., Li, Z., Naik, M., & Xing, E. P. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.191 · Cited by 23 · see citing works
ThinkSum: Probabilistic reasoning over sets using large language models
Ozturkler, B., Malkin, N., Wang, Z., & Jojić, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.68 · Cited by 13 · see citing works
Arithmetic Reasoning with LLM: Prolog Generation & Permutation
Yang, X., Chen, B., & Tam, Y.-C. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers) · doi:10.18653/v1/2024.naacl-short.61 · Cited by 12 · see citing works
Explicit Planning Helps Language Models in Logical Reasoning
Zhao, H., Wang, K., Yu, M., & Mei, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.688 · Cited by 11 · see citing works
Open-Ethical AI: Advancements in Open-Source Human-Centric Neural Language Models
Sicari, S., Moreno, J. F. C., Rizzardi, A., & Coen‐Porisini, A. (2024) · ACM Computing Surveys · doi:10.1145/3703454 · Cited by 10 · see citing works
DSPy-based neural-symbolic pipeline to enhance spatial reasoning in LLMs
Wang, R., & Sun, K. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108022 · Cited by 8 · see citing works
Integrating symbolic reasoning into neural generative models for design generation
Jacobson, M. J., & Xue, Y. (2024) · Artificial Intelligence · doi:10.1016/j.artint.2024.104257 · Cited by 8 · see citing works
Prompt engineering for bibliographic web-scraping
Ochando, M. B., Gutiérrez, J. J. P., & Ovalle-Perandones, M.-A. (2025) · Scientometrics · doi:10.1007/s11192-025-05372-5 · Cited by 7 · see citing works
Zhou, J., Pang, L., Shen, H., & Cheng, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.553 · Cited by 7 · see citing works
Enhancing consistency with the fusion of paralleled decoders for text generation
Li, Y., Huang, H., Bai, Y., & Gao, Y. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102652 · Cited by 7 · see citing works
Measuring Trustworthiness in Neuro-Symbolic Integration
Agiollo, A., & Omicini, A. (2023) · Annals of Computer Science and Information Systems · doi:10.15439/2023f6019 · Cited by 6 · see citing works
Disentangling Extraction and Reasoning in Multi-hop Spatial Reasoning
Mirzaee, R., & Kordjamshidi, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.221 · Cited by 5 · see citing works
Do language models have coherent mental models of everyday things?
Gu, Y., Mishra, B. D., & Clark, P. A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.106 · Cited by 5 · see citing works
Betz, G., & Richardson, K. (2023) · PLoS ONE · doi:10.1371/journal.pone.0281372 · Cited by 4 · see citing works
AnyTOD: A Programmable Task-Oriented Dialog System
Zhao, J., Cao, Y., Gupta, R., Lee, H., Rastogi, A., Wang, M., Soltau, H., Izhak Shafran, J., & Wu, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1006 · Cited by 4 · see citing works
Reliable Reasoning Beyond Natural Language
Borazjanizadeh, N., & Piantadosi, S. T. (2024) · arXiv · doi:10.48550/arxiv.2407.11373 · Cited by 3 · see citing works
“Similarly, Nye et al. [23] improves the performance of LLMs in story generation and instruction-following tasks by using a symbolic reasoning module to check the logical consistency of generated text against a minimal world model.”
Stone, W., Stone, S., & Lindstedt, J. K. (2024) · IEEE Western New York Image and Signal Processing Workshop (WNYISPW) · doi:10.1109/wnyispw63690.2024.10786497 · Cited by 2 · see citing works
Adaptive-TOD: An LLM-driven and adaptive agent for diverse interaction modes
Li, P., Yang, Q., Xu, S., Li, X., Li, Z., Chen, W., Liu, Y., Guo, T., Tang, J., & Wen, Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130991 · Cited by 2 · see citing works
Betz, G., & Richardson, K. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.900943 · Cited by 2 · see citing works
Decider: A Dual-System Rule-Controllable Decoding Framework for Language Generation
Xu, C., Lan, T., Ji, Y., Yu, C., Wang, W., Gao, J., Dong, Q., Qian, K., Li, P., Bi, W., et al. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3554819 · Cited by 2 · see citing works
Plug-and-Play Recipe Generation with Content Planning
Liu, Y., Su, Y., Shareghi, E., & Collier, N. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.19 · Cited by 2 · see citing works
Neurosymbolic Narrative Generation for Cultural Heritage
Palma, C. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230129 · Cited by 1 · see citing works
NEUROSTRUCTURAL DECODING: Neural Text Generation with Structural Constraints
Bastan, M., Surdeanu, M., & Balasubramanian, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.528 · Cited by 1 · see citing works
CoRRPUS: Code-based Structured Prompting for Neurosymbolic Story Understanding
Dong, Y., Martin, L. J., & Callison-Burch, C. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.832 · Cited by 1 · see citing works
A Minimal Model for Compositional Generalization on gSCAN
Hein, A., & Diepold, K. J. (2022) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2022.blackboxnlp-1.1 · Cited by 1 · see citing works
Artificial Intelligence and Language
Lenci, A., & Vestrucci, A. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-95504-1.00241-6 · Cited by 1 · see citing works
Alviano, M., Capalbo, M., Gottlob, G., Grillo, L., Kareem, I., Lo Scudo, F., Piccolo, S. A., & Rodriguez Reiners, L. A. (2026) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.21678
“According to (Nye et al. 2021; Ren et al. 2025), LLMs are better suited for data extraction than for complex reasoning, particularly in scenarios requiring grounded statements and logical inference.”
Arithmetic Reasoning with LLM: Prolog Generation & Permutation
Yang, X., Chen, B., & Tam, Y.-C. (2024) · arXiv · doi:10.48550/arxiv.2405.17893
“Neural Symbolic Reasoning Neural symbolic reasoning (Andreas et al., 2016; Neelakantan et al., 2017; Hudson and Manning, 2019; Gupta et al., 2020; Nye et al., 2021) aims to leverage both neural networks and symbolic reasoning to obtain better reasoning abilities and transparency.”
BackgroundAbstraction, Reasoning and Deep Learning: A Study of the "Look and Say" Sequence
Zadrozny, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.12755
For Generated Text, Is NLI-Neutral Text the Best Text?
Mersinias, M., & Mahowald, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.170
Case-Based Deduction for Entailment Tree Generation
Shi, J., Ding, X., & Liu, T. (2024) · Mathematics · doi:10.3390/math12182893
Enabling large language model with plug-and-play symbolic reasoning components
Galitsky, B. A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-30046-2.00014-4
- Authors
- Nye, M., Tessler, M. H., Tenenbaum, J. B., & Lake, B. M.
- Affiliations
- Massachusetts Institute of Technology (US)
- New York University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Volume
- 34
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2107.02794
- OpenAlex
- W3178308100
- Cited by
- 40
- References
- 39
- Citations by year
- 2025: 6 · 2024: 7 · 2023: 20 · 2022: 6 · 2021: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Keywords
- System 1 and System 2, logical consistency, coherence, neuro-symbolic reasoning, training-free reasoning, story generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
