Igniting Language Intelligence: The Hitchhiker’s Guide from Chain-of-Thought Reasoning to Language Agents
Open accessThe DOI matches its Crossref record; 241 of the 241 references it cites were found in a registry or an index.
Abstract
Large language models (LLMs) have dramatically enhanced the field of language intelligence, as demonstrably evidenced by their formidable empirical performance across a spectrum of complex reasoning tasks. Additionally, theoretical proofs have illuminated their emergent reasoning capabilities, providing a compelling showcase of their advanced cognitive abilities in linguistic contexts. Critical to their remarkable efficacy in handling complex reasoning tasks, LLMs leverage the intriguing chain-of-thought (CoT) reasoning techniques, obliging them to formulate intermediate steps en route to deriving an answer. The CoT reasoning approach has not only exhibited proficiency in amplifying reasoning performance but also in enhancing interpretability, controllability, and flexibility. In light of these merits, recent research endeavors have extended CoT reasoning methodologies to nurture the development of autonomous language agents, which adeptly adhere to language instructions and execute actions within varied environments. This survey article orchestrates a thorough discourse, penetrating vital research dimensions, encompassing (i) the foundational mechanics of CoT techniques, with a focus on elucidating the circumstances and justification behind its efficacy; (ii) the paradigm shift in CoT; and (iii) the burgeoning of language agents fortified by CoT approaches. Prospective research avenues envelop explorations into generalization, efficiency, customization, scaling, and safety. A repository for the related papers is available at https://github.com/Zoeyyao27/CoT-Igniting-Agent .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Furthermore, while recent agent frameworks adopt reasoning heuristics like Chain-of-Thought (CoT) [163,193] and ReAct [136] to simulate deliberative processes, these approaches remain shallow in semantic comprehension.”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [193]
In the citing paper
“these approaches remain shallow in semantic comprehension.”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [196]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Searle, J. R. (1969) · doi:10.1017/cbo9781139173438 · Cited by 11,718
Intelligent agents: theory and practice
Wooldridge, M., & Jennings, N. R. (1995) · The Knowledge Engineering Review · doi:10.1017/s0269888900008122 · Cited by 6,810 · cited 1×
“, I need to ...” substantially improves the action prediction accuracy.”
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254
Self-citationLarge language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1×
“Singhal et al. [128] presented MultiMedQA, a benchmark that combines six existing open question answering datasets, and HealthSearchQA, a new free-response dataset of medical questions searched online.”
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Wechsler Adult Intelligence Scale-III
Ryan, J. J., & Lopez, S. J. (2001) · Understanding Psychological Assessment · doi:10.1007/978-1-4615-1185-4_2 · Cited by 3,731
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.03109 · Cited by 3,159
Self-citationCommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
Talmor, A., Herzig, J., Lourie, N., & Berant, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1421 · Cited by 2,511
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1516 · Cited by 2,428
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · cited 7×
“However, due to computational constraints, most studies utilize only a limited number of log steps [98].”
InfluentialGenerative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
Self-citationA Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.18223 · Cited by 2,090
Self-citationLarge language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013
Large Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941
Self-citationA Survey on Large Language Model based Autonomous Agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11432 · Cited by 1,925
Self-citationImproved Baselines with Visual Instruction Tuning
Liu, H., Li, C., Li, Y., & Lee, Y. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02484 · Cited by 1,748
Improved Baselines with Visual Instruction Tuning
Liu, H., Li, C., Li, Y., & Lee, Y. J. (2023) · arXiv · doi:10.48550/arxiv.2310.03744 · Cited by 1,733
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441
Emergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., & Mordatch, I. (2023) · arXiv · doi:10.48550/arxiv.2305.14325 · Cited by 1,202
Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2305.10601 · Cited by 1,158
Toolformer: Language Models Can Teach Themselves to Use Tools
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2302.04761 · Cited by 1,150
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087 · cited 1×
“The gap between reasoning and action is bridged by combining interleaving thought, action, and observation [50, 126, 174].”
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083
Self-citationEncouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S., & Tu, Z. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.992 · Cited by 1,062
Self-citationThe Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972 · cited 4×
“However, computation overhead is still an obstacle when modeling multi-agent communications [162].”
InfluentialThe rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
Flamingo: a visual language model for few-shot learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1723 · Cited by 966
Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
Qin, C., Zhang, A., Zhang, Z., Chen, J., Yasunaga, M., & Yang, D. (2023) · arXiv · doi:10.48550/arxiv.2302.06476 · Cited by 869
Self-citationAnil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.10403 · Cited by 834
Teaching Large Language Models to Self-Debug
Chen, X., Lin, M., Schärli, N., & Zhou, D. (2023) · arXiv · doi:10.48550/arxiv.2304.05128 · Cited by 812
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 798
Augmenting large language models with chemistry tools
M. Bran, A., Cox, S., Schilter, O., Baldassari, C., White, A. D., & Schwaller, P. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00832-8 · Cited by 794
LaMDA: Language Models for Dialog Applications
Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778 · cited 1×
“Four model families (LaMDA [139], GPT [12], PaLM [4], and Llama 3 [27]) across varying model sizes are involved.”
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · arXiv · doi:10.48550/arxiv.2304.08485 · Cited by 682
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591 · cited 1×
“Furthermore, this research delineated several promising avenues for future exploration, including aspects related to generalization, efficiency, customization, scaling, and safety.”
Training Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575 · cited 2×
“1 Figure 3 shows the performance of different model sizes within the same family on the GSM8K [22] dataset, 2 comparing accuracy between using CoT prompts and direct prompts.”
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., & Chen, W. (2023) · arXiv · doi:10.48550/arxiv.2305.11738 · Cited by 561
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29720 · Cited by 556 · cited 3×
“MM-CoT incorporates language (text) and vision (images) modalities into a two-stage framework: rationale generation and answer inference.”
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.09687 · Cited by 556
AndroidInTheWild: A Large-Scale Dataset For Android Device Control
Rawles, C., Lillicrap, T., Rodriguez, D., Riva, O., & Lillicrap, T. (2023) · neural information processing systems · doi:10.52202/075280-2609 · Cited by 521
The Web as a Knowledge-Base for Answering Complex Questions
Talmor, A., & Berant, J. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1059 · Cited by 520
Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505
Self-citationIs ChatGPT a General-Purpose Natural Language Processing Task Solver?
Qin, C., Zhang, A., Zhang, Z., Chen, J., Yasunaga, M., & Yang, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.85 · Cited by 504
Self-citationChatDev: Communicative Agents for Software Development
Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.810 · Cited by 490
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Shen, Y., Song, K., Tan, X., Li, D., Lu, W., & Zhuang, Y. (2023) · arXiv · doi:10.48550/arxiv.2303.17580 · Cited by 477
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Shen, Y., Song, K., Tan, X., Li, D., Lu, W., & Zhuang, Y. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1657 · Cited by 475
Large Language Models as Commonsense Knowledge for Large-Scale Task PlanningPreprint
Zhao, Z., Lee, W. S., & Hsu, D. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2305.14078 · Cited by 440
Recurrent World Models Facilitate Policy Evolution
Ha, D., & Schmidhuber, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.01999 · Cited by 429
Towards Reasoning in Large Language Models: A Survey
Huang, J., & Chang, K. C.-C. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.67 · Cited by 428
MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots
Deng, G., Liu, Y., Li, Y., Wang, K., Zhang, Y., Li, Z., Wang, H., Zhang, T., & Liu, Y. (2023) · arXiv · doi:10.48550/arxiv.2307.08715 · Cited by 418
OpenAGI: When LLM Meets Domain ExpertsPreprint
Ge, Y.-Q., Hua, W.-Y., Mei, K., et al. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2304.04370 · Cited by 378
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.557 · Cited by 349
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022) · arXiv · doi:10.48550/arxiv.2212.10509 · Cited by 349 · cited 1×
“Retrieval augmentation has been shown so effective that has been regarded as a standard solution to alleviate the factuality drawback [140, 174].”
Learning Interactive Real-World Simulators
Yang, S., Du, Y., Ghasemipour, K., Tompson, J., Kaelbling, L., Schuurmans, D., & Abbeel, P. (2023) · arXiv · doi:10.48550/arxiv.2310.06114 · Cited by 346
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K.-W., & Lim, E.-P. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.147 · Cited by 344
Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems
Ling, W., Yogatama, D., Dyer, C., & Blunsom, P. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1015 · Cited by 325
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S., & Tu, Z. (2023) · arXiv · doi:10.48550/arxiv.2305.19118 · Cited by 320
Self-citationCAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
Li, G., Hammoud, H. A. A. K., Itani, H., Khizbullin, D., & Ghanem, B. (2023) · arXiv · doi:10.48550/arxiv.2303.17760 · Cited by 315
Camel: Communicative agents for" mind" exploration of large language model society
Li, G., Hammoud, H., Itani, H., Khizbullin, D., & Ghanem, B. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2264 · Cited by 314 · cited 4×
“Language agents have found extensive applications in practical fields such as engineering [56, 84, 103], natural sciences [11, 49, 78], and social sciences [2, 79].”
Influential2024 IEEE International Conference on Robotics and Automation (ICRA)
(2024) · doi:10.1109/icra57147.2024 · Cited by 313
Are NLP Models really able to Solve Simple Math Word Problems?
Patel, A., Bhattamishra, S., & Goyal, N. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.168 · Cited by 307
Fact-Checking Complex Claims with Program-Guided Reasoning
Pan, L., Wu, X., Lu, X., Luu, A. T., Wang, W. Y., Kan, M., & Nakov, P. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.386 · Cited by 291
Jailbroken: How Does LLM Safety Training Fail?
Wei, A., Haghtalab, N., & Steinhardt, J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-3508 · Cited by 280
Jailbroken: How Does LLM Safety Training Fail?
Wei, A., Haghtalab, N., & Steinhardt, J. (2023) · arXiv · doi:10.48550/arxiv.2307.02483 · Cited by 280
ViperGPT: Visual Inference via Python Execution for Reasoning
Surís, D., Menon, S., & Vondrick, C. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01092 · Cited by 273
WebGPT: Browser-assisted question-answering with human feedback
Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.09332 · Cited by 272 · cited 1×
“As language agents are exposed to multi-turn interactions in distinct environments possibly with operating tools, new safety risks may emerge at a systematic level [53, 168], ranging from instruction input, environment perception, and reasoning process, as well as tool use.”
Verbal and non-verbal intelligence changes in the teenage brain
Ramsden, S., Richardson, F. M., Josse, G., Thomas, M. S. C., Ellis, C., Shakeshaft, C., Seghier, M. L., & Price, C. J. (2011) · Nature · doi:10.1038/nature10514 · Cited by 248
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
Geva, M., Khashabi, D., Segal, E., Khot, T., Roth, D., & Berant, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00370 · Cited by 242
ProPILE: Probing Privacy Leakage in Large Language ModelsPreprint
Kim, S., Yun, S., Lee, H., Gubri, M., Yoon, S.-H., & Oh, S. J. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2307.01881 · Cited by 236
Language Is Not All You Need: Aligning Perception with Language Models
Huang, S., Dong, L., Wang, W., Hao, Y., Singhal, S., Ma, S., Lv, T., Cui, L., Mohammed, O. K., Patra, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.14045 · Cited by 235 · cited 6×
“More recent studies have also explored modeling the inputs of different modalities into the same vector space, thus resulting in any-to-any representation learning [46, 88, 160] and interleaved multimodal representation learning [57, 190].”
Influential(1995) · Lecture Notes in Computer Science · doi:10.1007/3-540-58855-8 · Cited by 232
Locating and Editing Factual Associations in GPT
Meng, K., Bau, D., Andonian, A., & Belinkov, Y. (2022) · arXiv · doi:10.48550/arxiv.2202.05262 · Cited by 230
Gorilla: Large Language Model Connected with Massive APIs
Patil, S. G., Zhang, T., Wang, X., & Gonzalez, J. E. (2023) · arXiv · doi:10.48550/arxiv.2305.15334 · Cited by 226 · cited 1×
“This includes everything from executing codes or queries through a JavaScript element selection on a webpage [196], executing programs via code interpreters or compilers [32, 34, 92, 115, 135], to interacting with online expert models which serve as callable APIs [29, 100, 124].”
Gorilla: Large Language Model Connected with Massive APIs
Patil, S., Zhang, T., Wang, X., & Gonzalez, J. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-4020 · Cited by 226
Natural Language Reasoning, A Survey
Yu, F., Zhang, H., Tiwari, P., & Wang, B. (2024) · ACM Computing Surveys · doi:10.1145/3664194 · Cited by 224
Voyager: An Open-Ended Embodied Agent with Large Language Models
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023) · arXiv · doi:10.48550/arxiv.2305.16291 · Cited by 222
You Only Look at Screens: Multimodal Chain-of-Action Agents
Zhang, Z., & Zhang, A. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2309.11436 · Cited by 222
Self-citationAgents that Reduce work and information Overload
Maes, P. (1995) · Readings in Human–Computer Interaction · doi:10.1016/b978-0-08-051574-8.50084-4 · Cited by 221
A Survey of Deep Learning for Mathematical Reasoning
Lu, P., Qiu, L., Yu, W., Welleck, S., & Chang, K.-W. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.817 · Cited by 221
AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., & Duan, N. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.149 · Cited by 211
Self-citationAGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2304.06364 · Cited by 211 · cited 1×
“In the era of LLMs, more comprehensive benchmark datasets have been released, such as MMLU [40], BIG-Bench [130], and AGI Eval [193].”
Self-citationMetaAgents: Large Language Model Based Agents for Decision-Making on Teaming
Li, Y., Sun, L., & Zhang, Y. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711032 · Cited by 195
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., & Zhou, J. (2023) · arXiv · doi:10.48550/arxiv.2308.12966 · Cited by 193
Zhou, A., Wang, K., Lu, Z., Shi, W., Luo, S., Qin, Z., Lu, S., Jia, A., Song, L., Zhan, M., et al. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2308.07921 · Cited by 193
Language Is Not All You Need: Aligning Perception with Language Models
Huang, S., Dong, L., Wang, W., Hao, Y., Singhal, S., Ma, S., Lv, T., Cui, L., Mohammed, O. K., Patra, B., et al. (2023) · neural information processing systems · doi:10.52202/075280-3155 · Cited by 191
Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages
Qin, L., Chen, Q., Wei, F., Huang, S., & Che, W. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.163 · Cited by 189
RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
Mandi, Z., Jain, S., & Song, S. (2023) · arXiv · doi:10.48550/arxiv.2307.04738 · Cited by 180
Reasoning with Language Model is Planning with World Model
Hao, S., Gu, Y., Ma, H., Hong, J., Wang, Z., Wang, D., & Hu, Z. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.507 · Cited by 168
RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
Mandi, Z., Jain, S., & Song, S. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610855 · Cited by 167
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
Zhang, R., Han, J., Liu, C., Gao, P., Zhou, A., Hu, X., Yan, S., Lu, P., Li, H., & Qiao, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.16199 · Cited by 166
Reasoning with Language Model is Planning with World Model
Hao, S., Gu, Y., Ma, H., Hong, J. J., Wang, Z., Wang, D. Z., & Hu, Z. (2023) · arXiv · doi:10.48550/arxiv.2305.14992 · Cited by 166
Towards Better Chain-of-Thought Prompting Strategies: A SurveyPreprint
Yu, Z., He, L., Wu, Z., Dai, X., & Chen, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.04959 · Cited by 162
Wang, Y., Zhang, Z., & Wang, R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.482 · Cited by 161
Self-citationNavigation with Large Language Models: Semantic Guesswork as a Heuristic for PlanningPreprint
Shah, D., Equi, M., Osiński, B., Xia, F., Ichter, B., & Levine, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.10103 · Cited by 154
Rationale-Augmented Ensembles in Language ModelsPreprint
Wang, X., Wei, J., Schuurmans, D., Le, Q. V., Ed, H., & Zhou, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.00747 · Cited by 152
Large Language Models as Optimizers
Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., & Chen, X. (2023) · arXiv · doi:10.48550/arxiv.2309.03409 · Cited by 151
Locating and Editing Factual Associations in GPT
Meng, K., Bau, D., Andonian, A., & Belinkov, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1262 · Cited by 149 · cited 1×
“ROME [85] and MEMIT [86] used the locate-and-edit method to correct wrong knowledge.”
Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels
Zhang, X., de Greef, L., Swearngin, A., White, S., Murray, K., Yu, L., Shan, Q., Nichols, J., Wu, J., Fleizach, C., et al. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445186 · Cited by 144
Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language ModelsPreprint
Yao, Y., Li, Z., & Zhao, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.16582 · Cited by 142
Self-citationAndroid in the Wild: A Large-Scale Dataset for Android Device Control
Rawles, C., Li, A., Rodriguez, D., Riva, O., & Lillicrap, T. (2023) · arXiv · doi:10.48550/arxiv.2307.10088 · Cited by 141
Character-LLM: A Trainable Agent for Role-Playing
Shao, Y., Li, L., Dai, J., & Qiu, X. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.814 · Cited by 140
Character-LLM: A Trainable Agent for Role-Playing
Shao, Y., Li, L., Dai, J., & Qiu, X. (2023) · arXiv · doi:10.48550/arxiv.2310.10158 · Cited by 140
Language Models can Solve Computer Tasks
Kim, G., Baldi, P., & McAleer, S. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1723 · Cited by 137
MASTERKEY: Automated Jailbreaking of Large Language Model Chatbots
Deng, G., Liu, Y., Li, Y., Wang, K., Zhang, Y., Li, Z., Wang, H., Zhang, T., & Liu, Y. (2024) · Proceedings 2024 Network and Distributed System Security Symposium · doi:10.14722/ndss.2024.24188 · Cited by 137
Evolutionary Reinforcement Learning: A Survey
Bai, H., Cheng, R., & Jin, Y. (2023) · Intelligent Computing · doi:10.34133/icomputing.0025 · Cited by 136
Reasoning with Language Model Prompting: A Survey
Qiao, S., Ou, Y., Zhang, N., Chen, X., Yao, Y., Deng, S., Tan, C., Huang, F., & Chen, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.294 · Cited by 134
Self-citationAnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model
Moon, S., Madotto, A., Lin, Z., Nagarajan, T., Smith, M., Jain, S., Yeh, C.-F., Murugesan, P., Heidari, P., Liu, Y., et al. (2023) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-industry.98 · Cited by 131
Teaching Small Language Models to Reason
Magister, L. C., Mallinson, J., Adamek, J., Malmi, E., & Severyn, A. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.151 · Cited by 127
OpenR: An Open Source Framework for Advanced Reasoning with Large Language ModelsPreprint
Wang, J., Fang, M., Wan, Z., Wen, M., Zhu, J., Liu, A., Gong, Z., Song, Y., Chen, L., Ni, L. M.-S., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.09671 · Cited by 127
Chen, W., Ma, X., Wang, X., & Cohen, W. W. (2022) · arXiv · doi:10.48550/arxiv.2211.12588 · Cited by 125
Self-citationAPI-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
Li, M., Zhao, Y., Yu, B., Song, F., Li, H., Yu, H., Li, Z., Huang, F., & Li, Y. (2023) · arXiv · doi:10.48550/arxiv.2304.08244 · Cited by 124 · cited 1×
“For example, the popular user interface agents designed for web operation [196] and mobile device control [187] may result in privacy leakage and permission abuse.”
API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
Li, M., Zhao, Y., Yu, B., Song, F., Li, H., Yu, H., Li, Z., Huang, F., & Li, Y. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.187 · Cited by 124 · cited 1×
“Tools coming into play include knowledge bases, search engines, code interpreters, online models, applications, databases, and even bespoke tools specially created for specific tasks, overcoming the constraints of generic APIs [13, 60, 119, 138, 197].”
ChemCrow: Augmenting large-language models with chemistry tools
Bran, A. M., Cox, S., Schilter, O., Baldassari, C., White, A. D., & Schwaller, P. (2023) · arXiv · doi:10.48550/arxiv.2304.05376 · Cited by 122
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
Wang, B., Min, S., Deng, X., Shen, J., Wu, Y., Zettlemoyer, L., & Sun, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.153 · Cited by 120
MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning
Zhao, H., Cai, Z., Si, S., Ma, X., An, K., Chen, L., Liu, Z., Wang, S., Han, W., & Chang, B. (2023) · arXiv · doi:10.48550/arxiv.2309.07915 · Cited by 120 · cited 1×
“More recent studies have also explored modeling the inputs of different modalities into the same vector space, thus resulting in any-to-any representation learning [46, 88, 160] and interleaved multimodal representation learning [57, 190].”
Self-citationLarge Language Models are Better Reasoners with Self-Verification
Weng, Y., Zhu, M., Xia, F., Li, B., He, S., Liu, S., Sun, B., Liu, K., & Zhao, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.167 · Cited by 110
Large Language Models are Better Reasoners with Self-Verification
Weng, Y., Zhu, M., Xia, F., Li, B., He, S., Liu, S., Sun, B., Liu, K., & Zhao, J. (2022) · arXiv · doi:10.48550/arxiv.2212.09561 · Cited by 110 · cited 2×
“In light of the swift advancements detailed previously, CoT We classify the memory modules into two main types: short-term memory and long-term memory.”
OpenAGI: When LLM Meets Domain Experts
Ge, Y., Hua, W., Mei, K., Li, Z., Tan, J., Xu, S., Li, Z., & Zhang, Y. (2023) · neural information processing systems · doi:10.52202/075280-0242 · Cited by 108
Language Models can Solve Computer Tasks
Kim, G., Baldi, P., & McAleer, S. (2023) · arXiv · doi:10.48550/arxiv.2303.17491 · Cited by 101
Prompt Injection attack against LLM-integrated Applications
Liu, Y., Deng, G., Li, Y., Wang, K., Wang, Z., Wang, X., Zhang, T., Liu, Y., Wang, H., Zheng, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05499 · Cited by 100
Playing repeated games with Large Language Models
Akata, E., Schulz, L., Coda-Forno, J., Oh, S. J., Bethge, M., & Schulz, E. (2023) · arXiv · doi:10.48550/arxiv.2305.16867 · Cited by 99
Multimodal Chain-of-Thought Reasoning in Language Models
Zhang, Z., Zhang, A., Li, M., Zhao, H. Y., Karypis, G., & Smola, A. J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.00923 · Cited by 97
Self-citationExploring Human-Like Translation Strategy with Large Language Models
He, Z., Liang, T., Jiao, W., Zhang, Z., Yang, Y., Wang, R., Tu, Z., Shi, S., & Wang, X. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00642 · Cited by 97
Self-citationLarge language models as commonsense knowledge for large-scale task planning
Zhao, Z., Lee, W. S., & Hsu, D. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1387 · Cited by 95
ControlLLM: Augment Language Models with Tools by Searching on Graphs
Liu, Z., Lai, Z., Gao, Z., Cui, E., Li, Z., Zhu, X., Lu, L., Chen, Q., Qiao, Y., Dai, Y., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73254-6_6 · Cited by 95
VIMA: General Robot Manipulation with Multimodal Prompts
Jiang, Y., Gupta, A., Zhang, Z., Wang, G., Dou, Y., Chen, Y., Fei-Fei, L., Anandkumar, A., Zhu, Y., & Fan, L. (2022) · arXiv · doi:10.48550/arxiv.2210.03094 · Cited by 95
Self-citationNExT-GPT: Any-to-Any Multimodal LLM
Wu, S., Fei, H., Qu, L., Ji, W., & Chua, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.05519 · Cited by 93
Visual Chain of Thought: Bridging Logical Gaps with Multimodal InfillingsPreprint
Rose, D., Himakunthala, V., Ouyang, A., He, R., Mei, A., Lu, Y., Saxon, M., Sonar, C., Mirza, D., & Wang, W. Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.02317 · Cited by 92
CodePlan: Repository-Level Coding using LLMs and Planning
Bairi, R., Sonwane, A., Kanade, A., C., V. D., Iyer, A., Parthasarathy, S., Rajamani, S., Ashok, B., & Shet, S. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3643757 · Cited by 92 · cited 1×
“The representative application scenarios of agents include autonomous control [48, 111], research [11, 78], programming [7], and interaction [98].”
CodePlan: Repository-level Coding using LLMs and Planning
Bairi, R., Sonwane, A., Kanade, A., C, V. D., Iyer, A., Parthasarathy, S., Rajamani, S., Ashok, B., & Shet, S. (2023) · arXiv · doi:10.48550/arxiv.2309.12499 · Cited by 92
Can Large Language Models Really Improve by Self-critiquing Their Own Plans?
Valmeekam, K., Marquez, M., & Kambhampati, S. (2023) · arXiv · doi:10.48550/arxiv.2310.08118 · Cited by 90 · cited 1×
“This iterative process aims to minimize mistakes and maximize accurate The Hitchhiker’s Guide from CoT Reasoning to Language Agents 206:19”
Inference Scaling for Long-Context Retrieval Augmented Generation
Yue, Z., Zhuang, H., Bai, A., Hui, K., Jagerman, R., Zeng, H., Qin, Z., Wang, D., Wang, X., & Bendersky, M. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.04343 · Cited by 89
Pan, L., Saxon, M., Xu, W., Nathani, D., Wang, X., & Wang, W. Y. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00660 · Cited by 86 · cited 1×
“For instance, integrating external tools such as search engines and calculators into the verification process has shown to be beneficial [18, 19, 95, 97].”
ChatDev: Communicative Agents for Software Development
Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.07924 · Cited by 81
AgentBench: Evaluating LLMs as Agents
Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.03688 · Cited by 79
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Shaikh, O., Zhang, H., Held, W., Bernstein, M., & Yang, D. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.244 · Cited by 76
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Shaikh, O., Zhang, H., Held, W., Bernstein, M., & Yang, D. (2022) · arXiv · doi:10.48550/arxiv.2212.08061 · Cited by 76
Mass-Editing Memory in a Transformer
Meng, K., Sharma, A. S., Andonian, A., Belinkov, Y., & Bau, D. (2022) · arXiv · doi:10.48550/arxiv.2210.07229 · Cited by 76
Emergent autonomous scientific research capabilities of large language models
Boiko, D. A., MacKnight, R., & Gomes, G. (2023) · arXiv · doi:10.48550/arxiv.2304.05332 · Cited by 72
Lemur: Harmonizing Natural Language and Code for Language Agents
Xu, Y., Su, H., Chen, X., Mi, B., Liu, Q., Shi, W., Hui, B., Zhou, F., Liu, Y., Xie, T., et al. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.06830 · Cited by 72
Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.16789 · Cited by 71
Self-citationExamining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate
Xiong, K., Ding, X., Cao, Y., Liu, T., & Qin, B. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.508 · Cited by 66
Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., & Cobbe, K. (2023) · arXiv · doi:10.48550/arxiv.2305.20050 · Cited by 65 · cited 2×
“…work of Wei et al. [154], the latest CoT reasoning techniques have been strengthened throughout the full stack of the reasoning process, such as multimodal perception [46, 114, 177, 188], automatic prompting [24, 189], reasoning verification [65, 69, 156], and consistency-based sampling [149, 150].”
Guan, L., Valmeekam, K., Sreedharan, S., & Kambhampati, S. (2023) · arXiv · doi:10.48550/arxiv.2305.14909 · Cited by 64
Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
Wang, C., Liu, X., Yue, Y., Tang, X., Zhang, T., Jiayang, C., Yao, Y., Gao, W., Hu, X., Qi, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.07521 · Cited by 61
Self-citationAutomatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data
Shum, K., Diao, S., & Zhang, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.811 · Cited by 60
GoT: Effective Graph-of-Thought Reasoning in Language Models
Yao, Y., Li, Z., & Zhao, H. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.183 · Cited by 58
Self-citationProPILE: Probing Privacy Leakage in Large Language Models
Kim, S., Yun, S., Lee, H., Gubri, M., Yoon, S., & Oh, S. J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-0911 · Cited by 57
Is there an intelligent agent in your future?
Hendler, J. (1999) · Nature · doi:10.1038/nature28027 · Cited by 56
SafetyBench: Evaluating the Safety of Large Language Models
Zhang, Z., Lei, L., Wu, L., Sun, R., Huang, Y., Long, C., Liu, X., Lei, X., Tang, J., & Huang, M. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.830 · Cited by 56
Self-citationImproving LLM Reasoning through Scaling Inference Computation with Collaborative VerificationPreprint
Liang, Z., Liu, Y., Niu, T., Zhang, X., Zhou, Y., & Yavuz, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.05318 · Cited by 56
ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
Xu, B., An, Y., Lin, J., Wang, Q., Zhou, C., Zhang, Y., & Mao, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.14688 · Cited by 55
Verify-and-edit: A knowledge-enhanced chain-of-thought framework
Zhao, R., Li, X., Joty, S., Qin, C., & Bing, L. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.320 · Cited by 55
Tab-CoT: Zero-shot Tabular Chain of Thought
Jin, Z., & Lü, W. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.651 · Cited by 55
Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP
Khattab, O., Santhanam, K., Li, X. L., Hall, D., Liang, P., Potts, C., & Zaharia, M. (2022) · arXiv · doi:10.48550/arxiv.2212.14024 · Cited by 52 · cited 3×
“For example, short-term memory can be modeled within an episode of a multi-step task, the chain of action history [187], or the rationales or sub-question in the last several hops of multi-hop question answering [50, 174].”
Active Prompting with Chain-of-Thought for Large Language Models
Diao, S., Wang, P., Lin, Y., Pan, R., Liu, X., & Zhang, T. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.73 · Cited by 52
Guan, L., Valmeekam, K., Sreedharan, S., & Subbarao Kambhampati, B. (2023) · neural information processing systems · doi:10.52202/075280-3459 · Cited by 52
LLM As DBAPreprint
Zhou, X., Li, G., & Liu, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.05481 · Cited by 51
SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks
Lin, B. Y., Fu, Y., Yang, K., Brahman, F., Huang, S., Bhagavatula, C., Ammanabrolu, P., Choi, Y., & Ren, X. (2023) · arXiv · doi:10.48550/arxiv.2305.17390 · Cited by 50
Chu, Z., Chen, J., Chen, Q., Yu, W., He, T., Wang, H., Peng, W., Liu, M., Qin, B., & Liu, T. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.65 · Cited by 50
SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks
Lin, B. Y., Fu, Y., Yang, K., Brahman, F., Huang, S., Bhagavatula, C., Ammanabrolu, P., Choi, Y., & Ren, X. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1034 · Cited by 50
Mehta, N., Teruel, M., Deng, X., Sanz, S. F., Awadallah, A. H., & Kiseleva, J. (2024) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2024.findings-eacl.87 · Cited by 49
Misusing Tools in Large Language Models With Visual Adversarial ExamplesPreprint
Fu, X., Wang, Z., Li, S., Gupta, R. K., Mireshghallah, N., Berg-Kirkpatrick, T., & Fernandes, E. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.03185 · Cited by 45
Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate
Xiong, K., Ding, X., Cao, Y., Liu, T., & Qin, B. (2023) · arXiv · doi:10.48550/arxiv.2305.11595 · Cited by 45 · cited 1×
“Auto-UI [187] was trained on the Android UI control domain, achieving stable performance as an autonomous agent.”
Self-Prompting Large Language Models for Open-Domain QAPreprint
Li, J., Zhang, Z., & Zhao, H. (2022) · arXiv.org · doi:10.48550/arxiv.2212.08635 · Cited by 45
Self-citationDeductive Verification of Chain-of-Thought Reasoning
Ling, Z., Fang, Y., Li, X., Huang, Z., Lee, M., Memisevic, R., & Su, H. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1580 · Cited by 44
Deductive Verification of Chain-of-Thought Reasoning
Ling, Z., Fang, Y., Li, X., Huang, Z., Lee, M., Memisevic, R., & Su, H. (2023) · arXiv · doi:10.48550/arxiv.2306.03872 · Cited by 44
EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education
Dan, Y., Lei, Z., Gu, Y., Li, Y., Yin, J., Lin, J., Ye, L., Tie, Z., Zhou, Y., Wang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.02773 · Cited by 41
Self-citationMa, Z., Mei, Y., & Su, Z. (2023) · arXiv · doi:10.48550/arxiv.2307.15810 · Cited by 40
PaLI-X: On Scaling up a Multilingual Vision and Language Model
Chen, X., Djolonga, J., Padlewski, P., Mustafa, B., Changpinyo, S., Wu, J., Riquelme, C., Goodman, S., Wang, X., Tay, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.18565 · Cited by 39
MetaAgents: Large Language Model Based Agents for Decision-Making on Teaming
Li, Y., Sun, L., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2310.06500 · Cited by 38
Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language ModelsPreprint
Lee, S., & Kim, G. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2306.06891 · Cited by 37
Investigate-Consolidate-Exploit: A General Strategy for Inter-Task Agent Self-EvolutionPreprint
Qian, C., Liang, S., Qin, Y., Ye, Y., Cong, X., Lin, Y., Wu, Y., Liu, Z., & Sun, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.13996 · Cited by 37
Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation
Ning, X., Lin, Z., Zhou, Z., Wang, Z., Yang, H., & Wang, Y. (2023) · arXiv · doi:10.48550/arxiv.2307.15337 · Cited by 36 · cited 1×
“Different from preceding works that focus on introducing structural information into CoT reasoning, Ning et al. [93] proposed SoT (Skeleton-of-Thought) to accelerate the CoT reasoning process.”
Self-citationZhu, X., Chen, Y., Tian, H., Tao, C., Su, W., Yang, C., Huang, G., Li, B., Lu, L., Wang, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.17144 · Cited by 33
Self-citationEffective Long-Context Scaling of Foundation Models
Xiong, W., Liu, J., Molybog, I., Zhang, H., Bhargava, P., Hou, R., Martin, L., Rungta, R., Sankararaman, K. A., Oguz, B., et al. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.260 · Cited by 33
Learn to Interpret Atari AgentsPreprint
Yang, Z., Bai, S., Zhang, L., & Torr, P. H. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.11276 · Cited by 33
WebArena: A Realistic Web Environment for Building Autonomous Agents
Zhou, S., Xu, F. F., Zhu, H., Zhou, X., Lo, R., Sridhar, A., Cheng, X., Ou, T., Bisk, Y., Fried, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.13854 · Cited by 31
Xu, L., Zhao, K., Zhu, L., & Xue, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.05818 · Cited by 30
Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word ProblemsPreprint
Wang, L., Yogatama, D., Dyer, C., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.04146 · Cited by 29
TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wild
Li, H., Li, S., Cai, D., et al. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2309.08637 · Cited by 29
When Do Program-of-Thoughts Work for Reasoning?Preprint
Bi, Z., Zhang, N.-Y., Jiang, Y.-N., Deng, S.-M., Zheng, G.-Z., & Chen, H.-J. (2023) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2308.15452 · Cited by 29
The nature of verbal comprehension
Sternberg, R. J., Powell, J. S., & Kaye, D. B. (1982) · Poetics · doi:10.1016/0304-422x(82)90031-6 · Cited by 28
Large language models as tool makers
Cai, T., Wang, X., Ma, T., Chen, X., & Zhou, D. (2023) · arXiv · doi:10.48550/arxiv.2305.17126 · Cited by 28
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
Snell, C., Lee, J., Xu, K., & Kumar, A. (2024) · arXiv · doi:10.48550/arxiv.2408.03314 · Cited by 27 · cited 1×
“…highlights the significance of inference scaling laws in optimizing reasoning performance via dynamically allocating computational resources, whether through iterative refinement [96, 144, 183], search-based verification [129], collaborative verification [64], or inference-time voting [17].”
Tutorial on Multimodal Machine Learning
Morency, L., Liang, P. P., & Zadeh, A. (2022) · doi:10.18653/v1/2022.naacl-tutorials.5 · Cited by 26
Jignasu, A., Marshall, K. A., Ganapathysubramanian, B., Balu, A., Hegde, C., & Krishnamurthy, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.02465 · Cited by 26
Chu, Z., Chen, J., Chen, Q., Yu, W., He, T., Wang, H., Peng, W., Liu, M., Qin, B., & Liu, T. (2023) · arXiv · doi:10.48550/arxiv.2309.15402 · Cited by 25
Research works printed but not linked: 32
- 10.5555/3618408.3618425 · doi:10.5555/3618408.3618425
- Zhen Bi, Ningyu Zhang, Yinuo Jiang, Shumin Deng, Guozhou Zheng, and Huajun Chen. 2024. When do program-of-thought works for reasoning? In Proceedings of the 38th AAAI Conference on Artificial Intelligence, the 36th Conference on Innovative Applications of Artificial Intelligence, and the 14th Symposium on Educational Advances in Artificial Intelligence (AAAI’24/IAAI’24/EAAI’24). 17691–17699.
- 10.5555/3692070.3692537 · doi:10.5555/3692070.3692537
- Zhibin Gou, Zhihong Shao, Yeyun Gong, yelong shen, Yujiu Yang, Minlie Huang, Nan Duan, and Weizhu Chen. 2024. ToRA: A tool-integrated reasoning agent for mathematical problem solving. In Proceedings of the 12th International Conference on Learning Representations (ICLR’24).
- Wes Gurnee and Max Tegmark. 2024. Language models represent space and time. In Proceedings of the 12th International Conference on Learning Representations (ICLR’24).
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In Proceedings of the 9th International Conference on Learning Representations (ICLR’21).
- Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the MATH dataset. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks.
- Sirui Hong, Mingchen Zhuge, Jiaqi Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. 2024. MetaGPT: Meta programming for a multi-agent collaborative framework. In Proceedings of the 12th International Conference on Learning Representations (ICLR’24).
- Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. Large language models cannot self-correct reasoning yet. In Proceedings of the 12th International Conference on Learning Representations (ICLR’24).
- Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning (ICML’23).
- Ansong Ni, Srini Iyer, Dragomir Radev, Ves Stoyanov, Wen-Tau Yih, Sida I. Wang, and Xi Victoria Lin. 2023. LEVER: Learning to verify language-to-code generation with execution. In Proceedings of the 40th International Conference on Machine Learning (ICML’23).
- Xuefei Ning, Zinan Lin, Zixuan Zhou, Zifu Wang, Huazhong Yang, and Yu Wang. 2023. Skeleton-of-thought: Large language models can do parallel decoding. In Proceedings of the Efficient Natural Language and Speech Processing Workshop (ENLSP-III).
- Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. 2022. Show your work: Scratchpads for intermediate computation with language models. In Proceedings of the Deep Learning for Code Workshop.
- 10.5555/3692070.3693649 · doi:10.5555/3692070.3693649
- Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Eleanor Jiang, Huajun Chen, et al. 2024. AutoAct: Automatic agent learning from scratch for QA via self-planning. In Proceedings of the ICLR 2024 Workshop on Large Language Model (LLM) Agents.
- Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2024. ToolLLM: Facilitating large language models to master 16000+ real-world APIs. In Proceedings of the 12th International Conference on Learning Representations (ICLR’24).
- Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning (ICML’21). 8748–8763.
- Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, and Desmond Elliott. 2023. Language modelling with pixels. In Proceedings of the 11th International Conference on Learning Representations (ICLR’23).
- Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. 2023. Language models are multilingual chain-of-thought reasoners. In Proceedings of the 11th International Conference on Learning Representations (ICLR’23).
- Srinivas Sunkara, Maria Wang, Lijuan Liu, Gilles Baechler, Yu-Chung Hsiao, Jindong Chen, Abhanshu Sharma, and James W. W. Stout. 2022. Towards better semantic understanding of mobile interfaces. In Proceedings of the 29th International Conference on Computational Linguistics (COLING’22). 5636–5650.
- Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Steven Zheng, et al. 2022. UL2: Unifying language learning paradigms. In Proceedings of the 10th International Conference on Learning Representations (ICLR’22).
- XAgent Team. 2023. XAgent: An autonomous agent for complex task solving. XAgent Blog.
- Karthik Valmeekam, Matthew Marquez, and Subbarao Kambhampati. 2023. Investigating the effectiveness of self-critiquing in LLMs solving planning tasks. In Proceedings of the NeurIPS 2023 Foundation Models for Decision Making Workshop.
- Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In Proceedings of the 11th International Conference on Learning Representations (ICLR’23).
- David E. Wilkins. 2014. Practical Planning: Extending the Classical AI Planning Paradigm. Elsevier.
- Hongqiu Wu, Zekai Xu, Tianyang Xu, Jiale Hong, Weiqi Wu, Hai Zhao, Min Zhang, and Zhezhi He. 2024. Scaling virtual world with delta-engine. arXiv preprint arXiv:2408.05842 (2024). · arXiv
- Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2022. An explanation of in-context learning as implicit Bayesian inference. In Proceedings of the 10th International Conference on Learning Representations (ICLR’22).
- Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2022. ReAct: Synergizing reasoning and acting in language models. In Proceedings of the 10th International Conference on Learning Representations (ICLR’22).
- Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, and Bill Yuchen Lin. 2024. Lumos: Learning agents with unified data, modular design, and open-source LLMs. In Proceedings of the ICLR 2024 Workshop on Large Language Model (LLM) Agents.
- 10.5555/3692070.3694642 · doi:10.5555/3692070.3694642
- Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2023. Large language models are human-level prompt engineers. In Proceedings of the 11th International Conference on Learning Representations (ICLR’23).
- Anni Zou, Zhuosheng Zhang, and Hai Zhao. 2024. AuRoRA: A one-for-all platform for augmented reasoning and refining with task-adaptive chain-of-thought prompting. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources, and Evaluation (LREC-COLING’24). 1801–1807.
Cited by
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Zhang, C., Wang, J., Wang, Z., Yau, S. K. S., Lin, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.00352 · Cited by 2,825
AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103599 · Cited by 671
“Furthermore, while recent agent frameworks adopt reasoning heuristics like Chain-of-Thought (CoT) [163,193] and ReAct [136] to simulate deliberative processes, these approaches remain shallow in semantic comprehension.”
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335
Empowering biomedical discovery with AI agents
Gao, S., Fang, A., Huang, Y., Giunchiglia, V., Noori, A., Schwarz, J. R., Ektefaie, Y., Kondic, J., & Zitnik, M. (2024) · Cell · doi:10.1016/j.cell.2024.09.022 · Cited by 315
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · arXiv · doi:10.48550/arxiv.2505.10468 · Cited by 295
Metadata
- Authors
- Zhang, Z., Yao, Y., Zhang, A., Tang, X., Ma, X., He, Z., Wang, Y., Gerstein, M., Wang, R., Liu, G., & Zhao, H.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Amazon (United States) (US)
- Yale University (US)
- ORCID
- Venue
- ACM Computing Surveys
- Published
- 2025-03-22
- Volume
- 57
- Issue
- 8
- Pages
- 1-39
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4407922851
- Cited by
- 51
- References
- 118
- Citations by year
- 2026: 24 · 2025: 22
- FWCI
- 18.45
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Computing Surveys (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62406188)
- Joint Funds of the National Natural Science Foundation of China (U21B2020)
- Natural Science Foundation of Shanghai (24ZR1440300)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- chain-of-thought reasoning, large language models, language agents, autonomous agents, emergent reasoning, interpretability, controllability, generalization, agent safety, scaling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.