CoChat
Sign Up
PreprintCited by 7Topic ModelingNatural Language Processing TechniquesLogic, Reasoning, and Knowledge

A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners

Open access
Jiang, Bowen, Xie, Yangxinyu, Hao, Zhuoqun, Wang, Xiaomeng, Mallick, Tanwi, Su, Weijie J., Taylor, Camillo J., Roth, Dan·2024·arXiv
DOI 10.48550/arxiv.2406.11050arXiv 2406.11050
Open this paper in CoChat — free accountOpen PDF

Abstract

This study introduces a hypothesis-testing framework to assess whether large language models (LLMs) possess genuine reasoning abilities or primarily depend on token bias. We go beyond evaluating LLMs on accuracy; rather, we aim to investigate their token bias in solving logical reasoning tasks. Specifically, we develop carefully controlled synthetic datasets, featuring conjunction fallacy and syllogistic problems. Our framework outlines a list of hypotheses where token biases are readily identifiable, with all null hypotheses assuming genuine reasoning capabilities of LLMs. The findings in this study suggest, with statistical guarantee, that most LLMs still struggle with logical reasoning. While they may perform well on classic problems, their success largely depends on recognizing superficial patterns with strong token bias, thereby raising concerns about their actual reasoning and generalization abilities. Codes and data are open-sourced at https://github.com/bowen-upenn/llm_token_bias.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2406.11050

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Randomizing the assignment of character names from a diverse pool minimizes sensitivity to tokenization problems and mitigates ethnic bias.

    “We randomize the assignment of character names from a diverse pool to minimize sensitivity to tokenization problems and mitigate ethnic bias (Navigli et al., 2023; Jiang et al., 2024).”

    From NoLiMa: Long-Context Evaluation Beyond Literal Matching (2025) · cites this paper as Jiang et al., 2024

    Section C.1 (page unavailable) describes generating biographies with random names and randomly sampled race, while Sections 3 and 3.4 examine how names can affect LLM reasoning. The paper does not say that assigning character names from a diverse pool minimizes tokenization problems or mitigates ethnic bias. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • The reasoning process in LLMs is probabilistic pattern-matching rather than formal reasoning.

    “Literature suggests that the reasoning process in LLMs is probabilistic pattern-matching rather than formal reasoning (Jiang et al., 2024).”

    From GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (2024) · cites this paper as Jiang et al., 2024

    The Discussion says LLMs primarily rely on token bias and may use semantic shortcuts instead of consistently applying reasoning. But it does not establish that their reasoning process is probabilistic pattern-matching rather than formal reasoning in general; the Limitations restrict the tasks examined, and the Discussion says the underlying mechanisms need further investigation. Page numbers are unavailable in our copy. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Performance can degrade under superficial changes that leave underlying logic intact, such as swapping surface values in GSM-style maths questions or reframing otherwise identical problems.

    “Performance can degrade under superficial changes that leave underlying logic intact-for example, swapping surface values in GSM-style maths questions[15] or reframing otherwise identical problems[16, 17].”

    From Illusions of reflection: open-ended task reveals systematic failures in Large Language Models' reflective reasoning (2025) · cites this paper as [17]

  • Jiang et al. show, with statistical guarantees, that most LLMs still struggle with logical reasoning due to strong token bias, where the reasoning output changes when a single token of input changes.

    “For instance, Jiang et al. (2024) show, with statistical guarantees, that most LLMs still struggle with logical reasoning due to strong token bias, where the reasoning output of the model changes when a single token of input changes.”

    From GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (2024) · cites this paper as Jiang et al., 2024

  • Small changes in input tokens can drastically alter model outputs, indicating strong token bias and high sensitivity and fragility.

    “Small changes in input tokens can drastically alter model outputs, indicating a strong token bias and suggesting that these models are highly sensitive and fragile (Jiang et al., 2024; Shi et al., 2023).”

    From GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (2024) · cites this paper as Jiang et al., 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 80

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. · arXiv
  2. Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. 2024. Large language models for mathematical reasoning: Progresses and challenges. arXiv preprint arXiv:2402.00157. · arXiv
  3. Yoav Benjamini and Yosef Hochberg. 1995. Control- ling the false discovery rate: A practical and pow- erful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1):289-300.
  4. Maciej Besta, Nils Blach, Ales Kubicek, Robert Gersten- berger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Pi- otr Nyczyk, et al. 2024. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17682-17690.
  5. Marcel Binz and Eric Schulz. 2023. Using cognitive psychology to understand gpt-3. Proceedings of the National Academy of Sciences, 120(6):e2218523120.
  6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901.
  7. Sébastien Bubeck, Varun Chandrasekaran, Ronen El- dan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lund- berg, et al. 2023. Sparks of artificial general intelli- gence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712. · arXiv
  8. Chengkun Cai, Xu Zhao, Yucheng Du, Haoliang Liu, and Lei Li. 2024. T2 of thoughts: Temperature tree elicits reasoning in large language models. arXiv preprint arXiv:2405.14075. · arXiv
  9. Xinyun Chen, Ryan A Chi, Xuezhi Wang, and Denny Zhou. 2024. Premise order matters in reason- ing with large language models. arXiv preprint arXiv:2402.08939. · arXiv
  10. Ishita Dasgupta, Andrew K Lampinen, Stephanie CY Chan, Antonia Creswell, Dharshan Kumaran, James L McClelland, and Felix Hill. 2022. Lan- guage models show human-like content effects on reasoning. arXiv preprint arXiv:2207.07051. · arXiv
  11. Jessica Echterhoff, Yao Liu, Abeer Alessa, Julian McAuley, and Zexue He. 2024. Cognitive bias in high-stakes decision-making with llms. arXiv preprint arXiv:2403.00811. · arXiv
  12. Yao Fu, Litu Ou, Mingyu Chen, Yuhao Wan, Hao Peng, and Tushar Khot. 2023. Chain-of-thought hub: A continuous effort to measure large language models' reasoning performance. arXiv preprint arXiv:2305.17306. · arXiv
  13. Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, et al. 2020. Evaluating models' local decision bound- aries via contrast sets. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1307-1323.
  14. Panagiotis Giadikiaroglou, Maria Lymperaiou, Giorgos Filandrianos, and Giorgos Stamou. 2024. Puzzle solving using reasoning of large language models: A survey. arXiv preprint arXiv:2402.11291. · arXiv
  15. Thilo Hagendorff, Sarah Fabi, and Michal Kosinski. 2023. Human-like intuitive behavior and reasoning biases emerged in large language models but disap- peared in chatgpt. Nature Computational Science, 3(10):833-838.
  16. Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992. · arXiv
  17. Reid Hastie and Robyn M Dawes. 2009. Rational choice in an uncertain world: The psychology of judgment and decision making. Sage Publications.
  18. Ruixin Hong, Hongming Zhang, Xinyu Pang, Dong Yu, and Changshui Zhang. 2023. A closer look at the self-verification abilities of large language models in logical reasoning. arXiv preprint arXiv:2311.07954. · arXiv
  19. Jie Huang and Kevin Chen-Chuan Chang. 2022. To- wards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403. · arXiv
  20. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xiny- ing Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798. · arXiv
  21. Mohsen Jamali, Ziv M Williams, and Jing Cai. 2023. Unveiling theory of mind in large language models: A parallel to single neurons in the human brain. arXiv preprint arXiv:2309.01660. · arXiv
  22. Albert Q Jiang, Alexandre Sablayrolles, Arthur Men- sch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guil- laume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. · arXiv
  23. Bowen Jiang, Yangxinyu Xie, Xiaomeng Wang, Wei- jie J Su, Camillo J Taylor, and Tanwi Mallick. 2024a. Multi-modal and multi-agent systems meet rational- ity: A survey. arXiv preprint arXiv:2406.00252. · arXiv
  24. Bowen Jiang, Zhijun Zhuang, Shreyas S. Shivakumar, Dan Roth, and Camillo J. Taylor. 2024b. Multi- agent vqa: Exploring multi-agent foundation models in zero-shot visual question answering. Preprint, arXiv:2403.14783. · arXiv
  25. Mingyu Jin, Qinkai Yu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, Mengnan Du, et al. 2024. The impact of reasoning step length on large language models. arXiv preprint arXiv:2401.04925. · arXiv
  26. Zhijing Jin, Abhinav Lalwani, Tejas Vaidhya, Xi- aoyu Shen, Yiwen Ding, Zhiheng Lyu, Mrinmaya Sachan, Rada Mihalcea, and Bernhard Schoelkopf. 2022. Logical fallacy detection. arXiv preprint arXiv:2202.13758. · arXiv
  27. Michal Kosinski. 2023. Evaluating large language mod- els in theory of mind tasks. arXiv e-prints, pages arXiv-2302.
  28. Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. 2024. Training language models to self- correct via reinforcement learning. arXiv preprint arXiv:2409.12917. · arXiv
  29. Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernan- dez, Dustin Li, Esin Durmus, Evan Hubinger, Jack- son Kernion, et al. 2023. Measuring faithful- ness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702. · arXiv
  30. Erich Leo Lehmann, Joseph P Romano, and George Casella. 1986. Testing statistical hypotheses, vol- ume 3. Springer.
  31. Bangzheng Li, Ben Zhou, Fei Wang, Xingyu Fu, Dan Roth, and Muhao Chen. 2023. Deceiving seman- tic shortcuts on reasoning chains: How far can models go without hallucination? arXiv preprint arXiv:2311.09702. · arXiv
  32. Xun Liang, Shichao Song, Zifan Zheng, Hanyu Wang, Qingchen Yu, Xunkai Li, Rong-Hua Li, Feiyu Xiong, and Zhiyu Li. 2024. Internal consistency and self- feedback in large language models: A survey. arXiv preprint arXiv:2407.14507. · arXiv
  33. Ruixi Lin and Hwee Tou Ng. 2023. Mind the bi- ases: Quantifying cognitive biases in language model prompting. In Findings of the Association for Com- putational Linguistics: ACL 2023, pages 5269-5281.
  34. Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. 2023. Faithful chain-of- thought reasoning. arXiv preprint arXiv:2301.13379. · arXiv
  35. Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2022. Z-icl: zero-shot in- context learning with pseudo-demonstrations. arXiv preprint arXiv:2212.09865. · arXiv
  36. Yingwei Ma, Yue Liu, Yue Yu, Yuanliang Zhang, Yu Jiang, Changjian Wang, and Shanshan Li. 2023. At which training stage does code data help llms reasoning? arXiv preprint arXiv:2309.16298. · arXiv
  37. Olivia Macmillan-Scott and Mirco Musolesi. 2024. (ir) rationality and cognitive biases in large language models. arXiv preprint arXiv:2402.09193. · arXiv
  38. Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153-157.
  39. William Merrill and Ashish Sabharwal. 2023. The expresssive power of transformers with chain of thought. arXiv preprint arXiv:2310.07923. · arXiv
  40. William Merrill and Ashish Sabharwal. 2024. A logic for expressing log-precision transformers. Advances in Neural Information Processing Systems, 36.
  41. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettle- moyer. 2022. Rethinking the role of demonstra- tions: What makes in-context learning work? arXiv preprint arXiv:2202.12837. · arXiv
  42. Melanie Mitchell and David C Krakauer. 2023. The de- bate over understanding in ai's large language models. Proceedings of the National Academy of Sciences, 120(13):e2215907120.
  43. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. A cor- pus and evaluation framework for deeper under- standing of commonsense stories. arXiv preprint arXiv:1604.01696. · arXiv
  44. Frederick Mosteller. 1952. Some statistical problems in measuring the subjective response to drugs. Biomet- rics, 8(3):220-226.
  45. Anirban Mukherjee and Hannah Hanwen Chang. 2024. Heuristic reasoning in ai: Instrumen- tal use and mimetic absorption. arXiv preprint arXiv:2403.09404. · arXiv
  46. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. Advances in neural in- formation processing systems, 35:27730-27744.
  47. Archiki Prasad, Alexander Koller, Mareike Hartmann, Peter Clark, Ashish Sabharwal, Mohit Bansal, and Tushar Khot. 2023. Adapt: As-needed decompo- sition and planning with language models. arXiv preprint arXiv:2311.05772. · arXiv
  48. Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. 2024. Agent q: Advanced reasoning and learning for autonomous ai agents. arXiv preprint arXiv:2408.07199. · arXiv
  49. Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. 2022. Reasoning with lan- guage model prompting: A survey. arXiv preprint arXiv:2212.09597. · arXiv
  50. Mathieu Ravaut, Bosheng Ding, Fangkai Jiao, Hailin Chen, Xingxuan Li, Ruochen Zhao, Chengwei Qin, Caiming Xiong, and Shafiq Joty. 2024. How much are llms contaminated? a comprehensive survey and the llmsanitize library. arXiv preprint arXiv:2404.00699. · arXiv
  51. Abigail See, Peter J Liu, and Christopher D Man- ning. 2017. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368. · arXiv
  52. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In Inter- national Conference on Machine Learning, pages 31210-31227.
  53. PMLR.
  54. Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Dur- rett. 2024. To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning. arXiv preprint arXiv:2409.12183. · arXiv
  55. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022a. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615. · arXiv
  56. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022b. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615. · arXiv
  57. Gaurav Suri, Lily R Slater, Ali Ziaee, and Morgan Nguyen. 2024. Do large language models show de- cision heuristics similar to humans? a case study using gpt-3.5. Journal of Experimental Psychology: General.
  58. Alaina N Talboy and Elizabeth Fuller. 2023. Challeng- ing the appearance of machine intelligence: Cogni- tive bias in llms. arXiv preprint arXiv:2304.01358. · arXiv
  59. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2018. Commonsenseqa: A question answering challenge targeting commonsense knowl- edge. arXiv preprint arXiv:1811.00937. · arXiv
  60. Alex Tamkin, Amanda Askell, Liane Lovitt, Esin Durmus, Nicholas Joseph, Shauna Kravec, Karina Nguyen, Jared Kaplan, and Deep Ganguli. 2023. Evaluating and mitigating discrimination in language model decisions. arXiv preprint arXiv:2312.03689. · arXiv
  61. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.13971. · arXiv
  62. Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. 2024. Language models don't always say what they think: unfaithful explanations in chain-of- thought prompting. Advances in Neural Information Processing Systems, 36.
  63. Amos Tversky and Daniel Kahneman. 1981. The fram- ing of decisions and the psychology of choice. sci- ence, 211(4481):453-458.
  64. Amos Tversky and Daniel Kahneman. 1983. Exten- sional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological re- view, 90(4):293.
  65. Amos Tversky and Daniel Kahneman. 1988. Ratio- nal choice and the framing of decisions. Decision making: Descriptive, normative, and prescriptive interactions, pages 167-192.
  66. Tomer Ullman. 2023. Large language models fail on trivial alterations to theory-of-mind tasks. arXiv preprint arXiv:2302.08399. USDL. 2024. Occupational employment and wage statistics. Accessed: 05-05-2024. · arXiv
  67. Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2022. Towards understanding chain-of-thought prompting: An empirical study of what matters. arXiv preprint arXiv:2212.10001. · arXiv
  68. Pengda Wang, Zilin Xiao, Hanjie Chen, and Freder- ick L Oswald. 2024. Will the real linda please stand up... to large language models? examining the representativeness heuristic in llms. arXiv preprint arXiv:2404.01461. · arXiv
  69. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022a. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682. · arXiv
  70. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022b. Chain-of-thought prompting elicits rea- soning in large language models. Advances in neural information processing systems, 35:24824-24837. Wikipedia contributors. 2024a. Forbes celebrity 100. Accessed: 2024-05-20.
  71. Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. 2024. Logicvista: Multimodal llm logical reason- ing benchmark in visual contexts. arXiv preprint arXiv:2407.04973. · arXiv
  72. Yangxinyu Xie, Tanwi Mallick, Joshua David Berg- erson, John K Hutchison, Duane R Verner, Jordan Branham, M Ross Alexander, Robert B Ross, Yan Feng, Leslie-Anne Levy, et al. 2024. Wildfiregpt: Tailored large language model for wildfire analysis. arXiv preprint arXiv:2402.07877. · arXiv
  73. Han Xu, Jingyang Ye, Yutong Li, and Haipeng Chen. 2024. Can speculative sampling accelerate react with- out compromising reasoning quality? In The Second Tiny Papers Track at ICLR 2024.
  74. Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. 2024. Do large language models latently perform multi-hop reasoning? arXiv preprint arXiv:2402.16837. · arXiv
  75. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Ben- gio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answer- ing. arXiv preprint arXiv:1809.09600. · arXiv
  76. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36.
  77. Zifan Zheng, Yezhaohui Wang, Yuxin Huang, Shichao Song, Bo Tang, Feiyu Xiong, and Zhiyu Li. 2024. Attention heads of large language models: A survey. arXiv preprint arXiv:2409.03752. · arXiv
  78. Ben Zhou, Kyle Richardson, Qiang Ning, Tushar Khot, Ashish Sabharwal, and Dan Roth. 2020. Temporal reasoning on implicit events from distant supervision. arXiv preprint arXiv:2010.12753. · arXiv
  79. Ben Zhou, Kyle Richardson, Xiaodong Yu, and Dan Roth. 2022. Learning to decompose: Hypothetical question decomposition based on comparable texts. arXiv preprint arXiv:2210.16865. · arXiv
  80. Kun Zhou, Yutao Zhu, Zhipeng Chen, Wentong Chen, Wayne Xin Zhao, Xu Chen, Yankai Lin, Ji-Rong Wen, and Jiawei Han. 2023. Don't make your llm an evaluation benchmark cheater. arXiv preprint arXiv:2311.01964. 7.967434 0.000000 True llama-2-70b-chat baseline 3 215 218 14.358452 0.000000 True llama-2-70b-chat zs-cot 30 199 229 11.167834 0.000000 True llama-2-70b-chat os 56 101 157 3.591391 0.000181 True llama-2-70b-chat os-cot 38 170 208 9.152553 0.000000 True llama-2-70b-chat fs 90 115 205 1.746076 0.042775 True llama-2-70b-chat fs-cot 43 150 193 7.702029 0.000000 True meta-llama-3-70b-instruct baseline 10 321 331 17.094106 0.000000 True meta-llama-3-70b-instruct zs-cot 12 296 308 16.182402 0.000000 True meta-llama-3-70b-instruct os 22 32 54 1.360828 0.090122 False meta-llama-3-70b-instruct os-cot 22 58 80 4.024922 0.000032 True meta-llama-3-70b-instruct fs 18 26 44 1.206045 0.116049 False meta-llama-3-70b-instruct fs-cot 19 44 63 3.149704 0.000883 True meta-llama-3-8b-instruct baseline 8 272 280 15.777018 0.000000 True meta-llama-3-8b-instruct zs-cot 5 263 268 15.759858 0.000000 True meta-llama-3-8b-instruct os 12 102 114 8.429272 0.000000 True meta-llama-3-8b-instruct os-cot 19 154 173 10.263860 0.000000 True meta-llama-3-8b-instruct fs 26 77 103 5.025179 0.000000 True Continued on next page 1 1.000000 1.000000 False meta-llama-3-70b-instruct baseline 3 9 12 1.732051 0.231876 False meta-llama-3-70b-instruct zs-cot 6 19 25 2.600000 0.022882 True meta-llama-3-70b-instruct os 4 20 24 3.265986 0.004696 True meta-llama-3-70b-instruct os-cot 7 23 30 2.921187 0.009415 True meta-llama-3-70b-instruct fs 0 31 31 5.567764 0.000000 True meta-llama-3-70b-instruct fs-cot 6 6 12 0.000000 1.000000 False meta-llama-3-8b-instruct baseline 7 32 39 4.003204 0.000337 True meta-llama-3-8b-instruct zs-cot 10 24 34 2.400980 0.038026 True meta-llama-3-8b-instruct os 5 35 40 4.743416 0.000019 True meta-llama-3-8b-instruct os-cot 4 24 28 3.779645 0.000707 True meta-llama-3-8b-instruct fs 8 29 37 3.452379 0.002308 True meta-llama-3-8b-instruct fs-cot 4 31 35 4.563833 0.000034 True claude-3-opus-20240229 baseline 12 26 38 2.271100 0.049984 True claude-3-opus-20240229 zs-cot 17 22 39 0.800641 0.586163 False claude-3-opus-20240229 os 10 16 26 1.176697 0.358975 False claude-3-opus-20240229 os-cot 11 11 22 0.000000 1.000000 False claude-3-opus-20240229 fs 6 9 15 0.774597 0.728687 False claude-3-opus-20240229 fs-cot 2 4 6 0.816497 0.773438 False claude-3-sonnet-20240229 baseline 3 23 26 3.922323 0.000431 True claude-3-sonnet-20240229 zs-cot 3 19 22 3.411211 0.003300 True claude-3-sonnet-20240229 os 8 18 26 1.961161 0.092843 False claude-3-sonnet-20240229 os-cot 6 17 23 2.293659 0.072048 False claude-3-sonnet-20240229 fs 10 17 27 1.347151 0.274523 False claude-3-sonnet-20240229 fs-cot 5 17 22 2.558409 0.038026 True mistral-large-latest baseline 3 6 9 1.000000 0.637718 False mistral-large-latest zs-cot 5 6 11 0.301511 1.000000 False mistral-large-latest os 4 11 15 1.807392 0.193859 False mistral-large-latest os-cot 5 22 27 3.271652 0.003609 True mistral-large-latest fs 2 6 8 1.414214 0.410773 False mistral-large-latest fs-cot 2 5 7 1.133893 0.596799 False · arXiv

Cited by

Sorted by citations

Authors
Jiang, B., Xie, Y., Hao, Z., Wang, X., Mallick, T., Su, W. J., Taylor, C. J., & Roth, D.
Affiliations
Argonne National Laboratory (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2406.11050
arXiv
2406.11050
OpenAlex
W4399794788
Cited by
2
References
0
Citations by year
2025: 1 · 2024: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (1934876, 2112665)
  • U.S. Department of Energy (DE‐ AC02‐06CH11357, 06CH11357)
  • Office of Science (06CH11357, DE-AC02-06CH11357)
  • Laboratory Directed Research and Development (DE‐AC02‐06CH11357)
  • Argonne National Laboratory (DE- AC02-06-CH11357, 06CH11357)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
Keywords
large language models, logical reasoning, conjunction fallacy, syllogistic reasoning, pattern recognition, generalization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review