On the Paradox of Learning to Reason from Data
Open accessAbstract
Logical reasoning is needed in a wide range of NLP tasks. Can a BERT model be trained end-to-end to solve logical reasoning problems presented in natural language? We attempt to answer this question in a confined problem space where there exists a set of parameters that perfectly simulates logical reasoning. We make observations that seem to contradict each other: BERT attains near-perfect accuracy on in-distribution test examples while failing to generalize to other data distributions over the exact same problem space. Our study provides an explanation for this paradox: instead of learning to emulate the correct reasoning function, BERT has in fact learned statistical features that inherently exist in logical reasoning problems. We also show that it is infeasible to jointly remove statistical features from data, illustrating the difficulty of learning to reason in general. Our result naturally extends to other neural models and unveils the fundamental difference between learning to reason and learning to achieve high performance on NLP benchmarks using statistical features.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The statistical nature of LLM training means they often fail to generalize appropriately to problems outside their training distribution.
“Moreover, the statistical nature of LLM training means they often fail to generalize appropriately to problems outside their training distribution (Zhang et al., 2022).”
From Reliable Reasoning Beyond Natural Language (2024) · cites this paper as Zhang et al., 2022
The Discussion and Section 4 report that models trained on one distribution of logical-reasoning problems fail on another because they rely on statistical features rather than the correct reasoning function. The study chiefly tests BERT in a confined problem space, with an extension to T5; it does not establish that LLMs generally or often fail on out-of-distribution problems. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
The complexity of theorem-proving procedures
Cook, S. A. (1971) · Proceedings of the third annual ACM symposium on Theory of computing - STOC '71 · doi:10.1145/800157.805047 · Cited by 6,385 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works
Learning Word Vectors for Sentiment Analysis
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011) · Cited by 3,710 · see citing works
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · see citing works
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · see citing works
Torralba, A., & Efros, A. A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995347 · Cited by 2,440 · see citing works
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · see citing works
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.215 · Cited by 1,781 · see citing works
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
McCoy, R. T., Pavlick, E., & Linzen, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1334 · Cited by 944 · see citing works
Annotation Artifacts in Natural Language Inference Data
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2017 · Cited by 907 · see citing works
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference
Zellers, R., Bisk, Y., Schwartz, R., & Choi, Y. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1009 · Cited by 712 · see citing works
Darwiche, A., & Marquis, P. (2002) · Cited by 602 · see citing works
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518 · see citing works
Don’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases
Clark, C., Yatskar, M., & Zettlemoyer, L. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1418 · Cited by 381 · see citing works
CommonsenseQA: A Question Answering Challenge Targeting Commonsense KnowledgePreprint
Talmor, A., Herzig, J., Lourie, N., & Berant, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00937 · Cited by 296 · see citing works
Analysing Mathematical Reasoning Abilities of Neural Models
Saxton, D., Grefenstette, E., Hill, F., & Kohli, P. (2019) · arXiv · doi:10.48550/arxiv.1904.01557 · Cited by 241 · see citing works
Learning a SAT Solver from Single-Bit SupervisionPreprint
Selsam, D., Lamm, M., Bünz, B., Liang, P., de Moura, L., & Dill, D. L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03685 · Cited by 213 · see citing works
LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
Jian, L., Cui, L., Liu, H., Huang, D., Wang, Y., & Zhang, Y. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/501 · Cited by 209 · see citing works
ReClor: A Reading Comprehension Dataset Requiring Logical ReasoningPreprint
Yu, W., Jiang, Z., Dong, Y., & Feng, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04326 · Cited by 179 · see citing works
Transformers as Soft Reasoners over Language
Clark, P., Tafjord, O., & Richardson, K. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/537 · Cited by 176 · see citing works
Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual
He, H., Zha, S., & Wang, H. (2019) · Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo) · doi:10.18653/v1/d19-6115 · Cited by 153 · see citing works
CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text
Sinha, K., Sodhani, S., Dong, J., Pineau, J., & Hamilton, W. L. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1458 · Cited by 110 · see citing works
What Can Neural Networks Reason About?Preprint
Xu, K., Li, J., Zhang, M., Du, S. S., Kawarabayashi, K., & Jegelka, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13211 · Cited by 107 · see citing works
Reasoning Over Paragraph Effects in Situations
Lin, K., Tafjord, O., Clark, P. E., & Gardner, M. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5808 · Cited by 92 · see citing works
SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solverPreprint
Wang, P.-W., Donti, P. L., Wilder, B., & Kolter, J. Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12149 · Cited by 85 · see citing works
Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning
Yin, D., Li, L. H., Hu, Z.-N., Peng, N., & Chang, K.-W. (2021) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.162 · Cited by 78 · see citing works
Self-citationOn Calibration and Out-of-domain GeneralizationPreprint
Wald, Y., Feder, A., Greenfeld, D., & Shalit, U. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10395 · Cited by 59 · see citing works
Measuring Systematic Generalization in Neural Proof Generation with TransformersPreprint
Gontier, N., Sinha, K., Reddy, S., & Pal, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.14786 · Cited by 37 · see citing works
Elazar, Y., Zhang, H., Goldberg, Y., & Roth, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.819 · Cited by 34 · see citing works
Self-citationOn the Robustness of Language Encoders against Grammatical Errors
Yin, F., Long, Q., Meng, T., & Chang, K.-W. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.310 · Cited by 31 · see citing works
Self-citationNaturalProofs: Mathematical Theorem Proving in Natural LanguagePreprint
Welleck, S., Liu, J., Bras, R. J. L., Hajishirzi, H., Choi, Y., & Cho, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.01112 · Cited by 25 · see citing works
Diagnosing the First-Order Logical Reasoning Ability Through LogicNLI
Tian, J., Li, Y., Chen, W., Xiao, L., He, H., & Jin, Y. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.303 · Cited by 24 · see citing works
Improving Graph Neural Network Representations of Logical Formulae with Subgraph PoolingPreprint
Crouse, M., Abdelaziz, I., Cornelio, C., Thost, V., Wu, L., Forbus, K. D., & Fokoue, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.06904 · Cited by 22 · see citing works
An Experimental Study of Formula Embeddings for Automated Theorem Proving in First-Order LogicPreprint
Abdelaziz, I., Thost, V., Crouse, M., & Fokoue, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.00423 · Cited by 3 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
1 of the 38 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- Nicolas Gontier, Koustuv Sinha, Siva Reddy, and Chris Pal. Measuring systematic gen- eralization in neural proof generation with transform- ers. Advances in Neural Information Processing Systems, 33:22231-22242, 2020.
- Christopher Hahn, Frederik Schmitt, Jens U. Kreber, Markus Norman Rabe, and Bernd Finkbeiner. Teaching temporal logics to neural networks. In ICLR. OpenReview.net, 2021.
- Khardon and Roth, 1997] Roni Khardon and Dan Roth. Learning to reason. Journal of the ACM (JACM), 44(5):697-725, 1997.
- Lample and Charton, 2020] Guillaume Lample and Franc ¸ois Charton. Deep learning for symbolic mathemat- ics. In ICLR. OpenReview.net, 2020.
- Alon Talmor, Oyvind Tafjord, Peter Clark, Yoav Goldberg, and Jonathan Berant. Leap-of- thought: Teaching pre-trained models to systematically reason over implicit knowledge. Advances in Neural In- formation Processing Systems, 33:20227-20237, 2020.
Cited by
Sorted by citations
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345 · see citing works
Generative AI in innovation and marketing processes: A roadmap of research opportunities
Cillo, P., & Rubera, G. (2024) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-024-01044-7 · Cited by 219 · see citing works
Krenn, M., Buffoni, L., Coutinho, B. C., Eppel, S., Foster, J. G., Gritsevskiy, A., Lee, H., Lu, Y., Moutinho, J. P., Sanjabi, N., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00735-0 · Cited by 96 · see citing works
Limitations of large language models in clinical problem-solving arising from inflexible reasoning
Kim, J., Podlasek, A., Shidara, K., Liu, F., Alaa, A., & Bernardo, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-22940-0 · Cited by 78 · see citing works
Natural Language Reasoning, A Survey
Yu, F., Zhang, H., Tiwari, P., & Wang, B. (2024) · ACM Computing Surveys · doi:10.1145/3664194 · Cited by 78 · see citing works
Self-citationHuppertz, M., Siepmann, R. M., Topp, D., Nikoubashman, O., Yüksel, C., Kuhl, C. K., Truhn, D., & Nebelung, S. (2024) · European Radiology · doi:10.1007/s00330-024-11115-6 · Cited by 65 · see citing works
Learning to Prompt in the Classroom to Understand AI Limits: A Pilot Study
Theophilou, E., Koyutürk, C., Yavari, M., Bursic, S., Donabauer, G., Telari, A., Testa, A., Boiano, R., Hernandez-Leo, D., Ruskov, M., et al. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-47546-7_33 · Cited by 58 · see citing works
GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
Mirzadeh, I., Alizadeh, K., Shahrokhi, H., Tuzel, O., Bengio, S., & Farajtabar, M. (2024) · arXiv · doi:10.48550/arxiv.2410.05229 · Cited by 57 · see citing works
“…reasoning process in LLMs is fragile and not formal (Kambhampati, 2024; Valmeekam et al., 2022; 2024; Nezhurina et al., 2024; McCoy et al., 2023; Zhang et al., 2023), even though it appears that these models understand symbols and can work with them to some limited degree (Boix-Adserà et al.,…”
Yiu, E., Kosoy, E., & Alison Gopnik, C. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231201401 · Cited by 61 · see citing works
LAMBADA: Backward Chaining for Automated Reasoning in Natural Language
Kazemi, M., Kim, N., Bhatia, D. S., Xu, X., & Ramachandran, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.361 · Cited by 34 · see citing works
Demystifying Chains, Trees, and Graphs of Thoughts
Besta, M., Memedi, F., Zhang, Z., Gerstenberger, R., Piao, G., Blach, N., Nyczyk, P., Copik, M., Kwaśniewski, G., Müller, J., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3598182 · Cited by 34 · see citing works
Natural Language Deduction through Search over Statement Compositions
Bostrom, K., Sprague, Z., Chaudhuri, S., & Durrett, G. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.358 · Cited by 20 · see citing works
The bewitching AI: The Illusion of Communication with Large Language Models
Bottazzi Grifoni, E., & Ferrario, R. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00893-6 · Cited by 15 · see citing works
Harnessing the Power of Large Language Models for Automated Code Generation and Verification
Antero, U., Blanco, F. O., Oñativia, J., Sallé, D., & Sierra, B. (2024) · Robotics · doi:10.3390/robotics13090137 · Cited by 12 · see citing works
Fodor, J., Deyne, S. D., & Suzuki, S. (2024) · Computational Linguistics · doi:10.1162/coli_a_00536 · Cited by 14 · see citing works
Walter, Y. (2023) · AI and Ethics · doi:10.1007/s43681-023-00276-7 · Cited by 11 · see citing works
MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logic
Sileo, D., & Lernould, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.303 · Cited by 10 · see citing works
Han, Z., Battaglia, F., Mansuria, K., Heyman, Y., & Terlecky, S. R. (2025) · AI · doi:10.3390/ai6010012 · Cited by 9 · see citing works
MindMap: Constructing Evidence Chains for Multi-Step Reasoning in Large Language Models
Wu, Y., Han, X., Song, W., Cheng, M., & Li, F. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29896 · Cited by 8 · see citing works
Explicit Planning Helps Language Models in Logical Reasoning
Zhao, H., Wang, K., Yu, M., & Mei, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.688 · Cited by 11 · see citing works
Silveira, I. C., Barbosa, A., da Costa, D. S. L., & Mauá, D. D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-79032-4_12 · Cited by 7 · see citing works
GPT-3 Models are Few-Shot Financial Reasoners
de Padua, R. S., Qureshi, I., & Karakaplan, M. U. (2023) · doi:10.5121/csit.2023.131216 · Cited by 7 · see citing works
RobustLR: A Diagnostic Benchmark for Evaluating Logical Robustness of Deductive Reasoners
Sanyal, S., Liao, Z., & Ren, X. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.653 · Cited by 11 · see citing works
A Simple, Yet Effective Approach to Finding Biases in Code Generation
Mouselinos, S., Malinowski, M., & Michalewski, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.718 · Cited by 8 · see citing works
LLM-Driven Knowledge Extraction in Temporal and Description Logics
Duranti, D., Giorgini, P., Mazzullo, A., Robol, M., & Roveri, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-77792-9_12 · Cited by 6 · see citing works
Model Checking Using Large Language Models—Evaluation and Future Directions
Batsakis, S., Tachmazidis, I., Mantle, M., Papadakis, N., & Antoniou, G. (2025) · Electronics · doi:10.3390/electronics14020401 · Cited by 5 · see citing works
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning
Sanyal, S., Yi‐chong, X., Wang, S., Yang, Z., Pryzant, R., Yu, W., Zhu, C., & Ren, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.347 · Cited by 6 · see citing works
You are what you're for: Essentialist categorization in large language modelsPreprint
Zhang, S., She, S., Gerstenberg, T., & Rosé, D. (2023) · doi:10.31234/osf.io/ypw5r · Cited by 5 · see citing works
LLMs and the Logical Space of Reasons
Sambrotta, M. (2025) · Minds and Machines · doi:10.1007/s11023-025-09751-y · Cited by 5 · see citing works
Wicke, P. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.302 · Cited by 4 · see citing works
Reliable Reasoning Beyond Natural Language
Borazjanizadeh, N., & Piantadosi, S. T. (2024) · arXiv · doi:10.48550/arxiv.2407.11373 · Cited by 3 · see citing works
“Moreover, the statistical nature of LLMs’ training and representation means they often fail in generalizing appropriately to problems outside their training distribution, especially in settings requiring reasoning and discrete processes [41].”
Large Language Model Reasoning Failures
Song, P., Han, P., & Goodman, N. (2026) · arXiv · doi:10.48550/arxiv.2602.06176 · Cited by 3 · see citing works
Assessing Logical Reasoning Capabilities of Encoder-Only Transformer Models
Pirozelli, P., José, M. M., Filho, P. D. T. P. D. C., Brandão, A. A. F., & Cozman, F. G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-71167-1_2 · Cited by 2 · see citing works
Modeling Multiple Adverse Pregnancy Outcomes: Learning from Diverse Data Sources
Mathur, S., Gadekar, V. P., Ramola, R., Wang, P., Thiruvengadam, R., Haas, D. M., Bhatnagar, S., Wadhwa, N., GARBH-Ini Study Group, Predrag Radivojac, A., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-66538-7_29 · Cited by 2 · see citing works
Can Language Models Trained on Written Monologue Learn to Predict Spoken Dialogue?
Umair, M., Mertens, J. B., Warnke, L., & de Ruiter, J. P. (2024) · Cognitive Science · doi:10.1111/cogs.70013 · Cited by 3 · see citing works
Are Large Language Models Good At Fuzzy Reasoning?
Singh, S. (2024) · International Conference on Computational Intelligence and Intelligent Systems (CIIS) · doi:10.1145/3708778.3708779 · Cited by 1 · see citing works
Toward Enhancing Reasoning Capabilities of LLMs: An Approach via Synthetic Logic Corpus
Morishita, T., Morio, G., Yamaguchi, A., & Sogawa, Y. (2025) · Journal of Natural Language Processing · doi:10.5715/jnlp.32.520 · Cited by 1 · see citing works
On the Relevance of Logic for Artificial Intelligence, and the Promise of Neurosymbolic Learning
Belle, V. (2025) · Neurosymbolic Artificial Intelligence · doi:10.1177/29498732251339951 · Cited by 1 · see citing works
Wahidi, M., Rizk, A., & Imad, R. (2025) · IEEE Access · doi:10.1109/access.2025.3589514 · Cited by 1 · see citing works
REMEND: Neural Decompilation for Reverse Engineering Math Equations from Binary Executables
Udeshi, M., Krishnamurthy, P., Karri, R., & Khorrami, F. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3749988 · Cited by 1 · see citing works
Can Large Language Models Generate Effective Datasets for Emotion Recognition in Conversations?
Kaplan, B. C., Carneiro, H., & Wermter, S. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.07.145 · Cited by 1 · see citing works
AI in Conflict Resolution: Practical Considerations, Opportunities and Challenges
Molak, A. (2025) · Conflict Resolution Quarterly · doi:10.1002/crq.70012 · Cited by 1 · see citing works
Mocanu, I. G., & Belle, V. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100036 · Cited by 1 · see citing works
Transformers in the Service of Description Logic-Based Contexts
Poulis, A., Tsalapati, E., & Koubarakis, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-77792-9_20 · Cited by 1 · see citing works
Can Large Language Models Act as Symbolic Reasoners?
Sullivan, R., & Elsayed, N. (2025) · IEEE International Conference on Computing and Machine Intelligence (ICMI) · doi:10.1109/icmi65310.2025.11141039 · Cited by 1 · see citing works
Artificial Intelligence: A Kalam and Sufi PerspectivePreprint
Khalili, A. M. (2023) · Preprints.org · doi:10.20944/preprints202306.0363.v1 · Cited by 1 · see citing works
Профессиональное самоопределение как компонент профессионального становления личности
Николаевна, Д. Т., & Николаевна, Ю. С. (2013) · Научное обеспечение системы повышения квалификации кадров
Artificial Intelligence: A Kalam and Sufi PerspectivePreprint
Khalili, A. M. (2023) · Preprints.org · doi:10.20944/preprints202306.0363.v2
Artificial Intelligence: A Kalam and Sufi PerspectivePreprint
Khalili, A. M. (2023) · Preprints.org · doi:10.20944/preprints202306.0363.v3
Withdrawn: Artificial Intelligence: A Kalam and Sufi PerspectivePreprint
Khalili, A. M. (2023) · Preprints.org · doi:10.20944/preprints202306.0363.v4
Artificial Intelligence: A Kalam and Sufi PerspectivePreprint
Khalili, A. (2024) · Preprints.org · doi:10.20944/preprints202306.0363.v6
ChemAlgebra: Algebraic Reasoning on Chemical Reactions
Valenti, A., Bacciu, D., & Vergari, A. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650037
Reasoning in Transformers – Mitigating Spurious Correlations and Reasoning Shortcuts
Enström, D., Kjellberg, V., & Johansson, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-71170-1_18
Model Checking with Large Language Models - Initial Experiments and Future Directions
Batsakis, S., Tachmazidis, I., Mantle, M., Papadakis, N., & Antoniou, G. (2024) · International Conference on Information, Intelligence, Systems & Applications (IISA) · doi:10.1109/iisa62523.2024.10786718
Reasoning Capabilities and Invariability of Large Language Models
Raganato, A., Peñaloza, R., Viviani, M., & Pasi, G. (2024) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat62293.2024.00025
A Learning and Reasoning Manifesto
Raedt, L. D., Heintz, F., Kristian Kersting, B., & Marra, G. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-03028-3_6
Wang, Y., & Chen, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-11442-6_20
SocraSynth: Adversarial Multi-LLM Reasoning
Chang, E. Y. (2025) · ACM eBooks · doi:10.1145/3749421.3749430
Jung, S., Choi, S.-H., Jeon, G., Cho, M., & Lim, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37574-2_11
Flach, J. M., Moreira, Á., & Lamb, L. C. (2026) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2026.6101
BAP-DL: A Deep Learning Approach for Binding Affinity Prediction
Abbassi, O., Kharmoum, N., Ziti, S., Amri, H. E., & Rhalem, W. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-30563-3_44
- Authors
- Zhang, H., Li, L. H., Meng, T., Chang, K.-W., & Broeck, G. V. D.
- Affiliations
- University of California, Los Angeles (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- Pages
- 3365-3373
- arXiv
- 2205.11502
- OpenAlex
- W4385767671
- Cited by
- 58
- References
- 38
- Citations by year
- 2026: 3 · 2025: 17 · 2024: 18 · 2023: 17 · 2022: 2 · 2013: 1
- FWCI
- 9.74
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1956441, 1943641, 1837129)
- Cisco Systems
- Defense Advanced Research Projects Agency
- Samsung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- logical reasoning, BERT, statistical features, generalization, natural language reasoning, T5, benchmark performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
