Reason from Fallacy: Enhancing Large Language Models’ Logical Reasoning through Logical Fallacy Understanding
Open accessAbstract
Large Language Models (LLMs) have demonstrated good performance in many reasoning tasks, but they still struggle with some complicated reasoning tasks including logical reasoning.One non-negligible reason for LLMs' suboptimal performance on logical reasoning is their overlooking of understanding logical fallacies correctly.To evaluate LLMs' capability of logical fallacy understanding (LFU), we propose five concrete tasks from three cognitive dimensions of WHAT, WHY, and HOW in this paper.Towards these LFU tasks, we have successfully constructed a new dataset LFUD based on GPT-4 accompanied by a little human effort.Our extensive experiments justify that our LFUD can be used not only to evaluate LLMs' LFU capability, but also to fine-tune LLMs to obtain significantly enhanced performance on logical reasoning.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLM agents were influenced by fallacious arguments.
“These agents were also influenced by such fallacious arguments, consistent with prior work showing susceptibility of language models to faulty reasoning [42, 58, 59].”
From Selective agreement, not sycophancy: investigating opinion dynamics in LLM interactions (2025) · cites this paper as [58]
Cites
Still being checked.
Cited by
Sorted by citations
Selective agreement, not sycophancy: investigating opinion dynamics in LLM interactions
Cau, E., Pansanella, V., Pedreschi, D., & Rossetti, G. (2025) · EPJ Data Science · doi:10.1140/epjds/s13688-025-00579-1 · Cited by 9 · see citing works
A Logical Fallacy-Informed Framework for Argument Generation
Mouchel, L., Paul, D., Cui, S., West, R., Bosselut, A., & Faltings, B. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.374 · Cited by 7 · see citing works
Skeptik: A Hybrid Framework for Combating Potential Misinformation in Journalism
Fan, A., Lei, F., Corman, S. R., & Maciejewski, R. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3766891 · Cited by 1 · see citing works
Zheng, X., Li, N., Luan, X., Wang, K., Shi, L., Sun, M., & Wang, H. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3787820 · Cited by 2 · see citing works
Huang, P.-C., Liu, C., Hsu, C., Chen, C.-S., Lee, P.-J., & Kang, Y. (2026) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri69576.2026.00074
Hua, W., Fang, Q., Sun, M., Xu, S., & Zhang, M. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02499-0
- Authors
- Li, Y., Wang, D., Liang, J., Jiang, G., He, Q., Xiao, Y., & Yang, D.
- Affiliations
- Fudan University (CN)
- Venue
- Findings of the Association for Computational Linguistics: NAACL 2024
- Published
- 2024
- Pages
- 3053-3066
- OpenAlex
- W4401043130
- Cited by
- 6
- References
- 0
- Citations by year
- 2026: 4 · 2025: 2
- FWCI
- 1.73
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: NAACL (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Fudan University
- Major Research Plan
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- large language models, logical reasoning, LLM fine-tuning, reasoning datasets, GPT-4, cognitive dimension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
