Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Open accessAbstract
We introduce Graph of Thoughts (GoT): a framework that advances prompting capabilities in large language models (LLMs) beyond those offered by paradigms such as Chain-of-Thought or Tree of Thoughts (ToT). The key idea and primary advantage of GoT is the ability to model the information generated by an LLM as an arbitrary graph, where units of information ("LLM thoughts") are vertices, and edges correspond to dependencies between these vertices. This approach enables combining arbitrary LLM thoughts into synergistic outcomes, distilling the essence of whole networks of thoughts, or enhancing thoughts using feedback loops. We illustrate that GoT offers advantages over state of the art on different tasks, for example increasing the quality of sorting by 62% over ToT, while simultaneously reducing costs by >31%. We ensure that GoT is extensible with new thought transformations and thus can be used to spearhead new prompting schemes. This work brings the LLM reasoning closer to human thinking or brain mechanisms such as recurrence, both of which form complex networks
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Graph-of-Thoughts (GoT) models reasoning as arbitrary graphs with thoughts as vertices and dependencies as edges, improving quality by 62% and reducing costs by 31% compared to ToT.
“Graph-of-Thoughts (GoT) models reasoning as arbitrary graphs with thoughts as vertices and dependencies as edges, improving quality by 62% and reducing costs by 31% compared to ToT [69, 832, 1376].”
From A Survey of Context Engineering for Large Language Models (2025) · cites this paper as [69]
The Introduction describes GoT as a graph with thoughts as vertices and dependencies as edges. Section 6.2 reports approximately 62% lower median error and more than 31% lower cost than ToT, but those figures are for a sorting task with P = 128; the claim omits that condition and presents the improvement as general. (LLM, conf=0.88)
Experimental — accuracy not yet measuredCoT and ToT reasoning extend single-agent reflection through structured deliberation.
“Methods such as Chain-of-Thought (CoT) and Tree-of-Thought (ToT) reasoning (Yao et al., 2023; Besta et al., 2024) extend single-agent reflection through structured deliberation”
From SwarmSys: Decentralized Swarm-Inspired Agents for Scalable and Adaptive Reasoning (2025) · cites this paper as Besta et al., 2024
The Introduction (page not available) describes CoT as a chain of intermediate reasoning steps and ToT as a tree that permits exploring different paths and backtracking. That supports structured deliberation, but the paper does not say CoT and ToT extend single-agent reflection; its Prompting Paradigms & Approaches section discusses self-reflection separately. (LLM, conf=0.78)
Experimental — accuracy not yet measuredGoT expands the tree-like reasoning structure in ToT to graph structures, resulting in more powerful prompting strategies.
“In GoT [52], the authors expand the tree-like reasoning structure in ToT to graph structures, resulting in more powerful prompting strategies.”
From A survey on large language model based autonomous agents (2024) · cites this paper as [52]
Tree-of-thoughts (ToT) and graph-of-thoughts (GoT) enable LLMs to branch, evaluate, and backtrack across multiple reasoning paths.
“Early approaches extend CoT into treeor graph-structured representations, such as tree-of-thoughts (ToT) [254] and graph-ofthoughts (GoT) [255], enabling LLMs to branch, evaluate, and backtrack across multiple reasoning paths.”
From A Survey of Large Language Models (2026) · cites this paper as [255]
Graph-of-Thoughts generalizes chain-of-thought reasoning into graph-structured trajectories, enabling intermediate states to branch, merge, and be reused.
“Early formulations such as Graph-of-Thoughts [74] generalize chain-ofthought reasoning into graph-structured trajectories, enabling intermediate states to branch, merge, and be reused.”
From Code as Agent Harness (2026) · cites this paper as [74]
Graph of Thoughts (GoT) prompting enables reasoning over a graph structure, where each node represents an intermediate thought, and the edges encode the dependencies between them.
“Building on CoT and ToT, Graph of Thoughts (GoT) (Besta et al., 2024) prompting enables reasoning over a graph structure, where each node represents an intermediate thought, and the edges encode the dependencies between them.”
From The AI Hippocampus: How Far are We From Human Memory? (2026) · cites this paper as Besta et al., 2024
Graph-of-Thoughts organises intermediate thoughts as nodes in a graph and applies graph-based operations to support flexible reasoning and information flow.
“Graph-of-Thoughts (Besta et al., 2024) organises intermediate thoughts as nodes in a graph and applies graph-based operations to support flexible reasoning and information flow.”
From A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems (2025) · cites this paper as Besta et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582 · see citing works
Graph neural networks: A review of methods and applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020) · AI Open · doi:10.1016/j.aiopen.2021.01.001 · Cited by 6,880 · see citing works
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
Geometric Deep Learning: Going beyond Euclidean data
Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2693418 · Cited by 3,661 · see citing works
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. (2020) · UTS ePRESS (University of Technology Sydney) · Cited by 3,174 · see citing works
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · see citing works
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · see citing works
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473 · see citing works
Representation Learning on Graphs: Methods and ApplicationsPreprint
Hamilton, W. L., Ying, R., & Leskovec, J. (2017) · IEEE Data Engineering Bulletin · doi:10.48550/arxiv.1709.05584 · Cited by 2,203 · see citing works
Deep Learning on Graphs: A Survey
Zhang, Z., Cui, P., & Zhu, W. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2981333 · Cited by 1,729 · see citing works
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526 · see citing works
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.346 · Cited by 1,279 · see citing works
Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2305.10601 · Cited by 1,158 · see citing works
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087 · see citing works
Hierarchical Models in the Brain
Friston, K. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000211 · Cited by 952 · cited 2× · see citing works
“Similarly, brains form complex networks, with graph-like patterns such as recurrence [28].”
BackgroundRhizoDB: A Bounded Activation-Flow Architecture for Graph-Based Memory Systems
Guberte, J. A. (2026) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.20695702 · Cited by 897 · see citing works
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829 · see citing works
Complexity-Based Prompting for Multi-Step ReasoningPreprint
Fu, Y., Peng, H.-C., Sabharwal, A., Clark, P., & Khot, T. (2022) · International Conference on Learning Representations · doi:10.48550/arxiv.2210.00720 · Cited by 670 · see citing works
ReAct: Synergizing Reasoning and Acting in Language Models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.03629 · Cited by 608 · see citing works
Eliciting Knowledge from Language Models Using Automatically Generated PromptsPreprint
Shin, T., Razeghi, Y., Logan, R. L., IV, Wallace, E., & Singh, S. (2020) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2010.15980 · Cited by 584 · see citing works
Wu, T., Terry, M. A., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517582 · Cited by 536 · see citing works
CHALLENGES IN PARALLEL GRAPH PROCESSING
Lumsdaine, A., Gregor, D., Hendrickson, B., et al. (2007) · Parallel Processing Letters · doi:10.1142/s0129626407002843 · Cited by 474 · see citing works
Large Language Model Guided Tree-of-ThoughtPreprint
Long, J.-Y. (2023) · arXiv.org · doi:10.48550/arxiv.2305.08291 · Cited by 350 · see citing works
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K.-W., & Lim, E.-P. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.147 · Cited by 344 · see citing works
Large Language Models Are Human-Level Prompt Engineers
Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., & Ba, J. (2022) · arXiv · doi:10.48550/arxiv.2211.01910 · Cited by 330 · see citing works
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
Huang, W., Abbeel, P., Pathak, D., & Mordatch, I. (2022) · arXiv · doi:10.48550/arxiv.2201.07207 · Cited by 304 · see citing works
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · arXiv · doi:10.48550/arxiv.2101.00190 · Cited by 289 · see citing works
Inner Monologue: Embodied Reasoning through Planning with Language Models
Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05608 · Cited by 287 · see citing works
Planning with Large Language Models for Code GenerationPreprint
Zhang, S., Chen, Z., Shen, Y., Ding, M., Tenenbaum, J., & Gan, C. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2303.05510 · Cited by 277 · see citing works
Graph Databases: New Opportunities for Connected Data
Robinson, I., Webber, J., & Eifrém, E. (2015) · CERN Document Server (European Organization for Nuclear Research) · Cited by 247 · see citing works
Machine Learning on Graphs: A Model and Comprehensive TaxonomyPreprint
Chami, I., Abu-El-Haija, S., Perozzi, B., Ré, C., & Murphy, K. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.03675 · Cited by 228 · see citing works
Self-Refine: Iterative Refinement with Self-Feedback
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.17651 · Cited by 219 · see citing works
Reasoning with Language Model Prompting: A Survey
Qiao, S., Ou, Y., Zhang, N., Chen, X., Yao, Y., Deng, S., Tan, C., Huang, F., & Chen, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.294 · Cited by 218 · see citing works
PromptChainer: Chaining Large Language Model Prompts through Visual Programming
Wu, T., Jiang, E., Donsbach, A., Gray, J., Molina, A., Terry, M., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519729 · Cited by 198 · see citing works
STaR: Bootstrapping Reasoning With Reasoning
Zelikman, E., Wu, Y., Mu, J., & Goodman, N. D. (2022) · arXiv · doi:10.48550/arxiv.2203.14465 · Cited by 182 · see citing works
Cheng, J., Yu, J. X., Ding, B., Yu, P. S., & Wang, H. (2008) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2008.4497500 · Cited by 167 · see citing works
Show Your Work: Scratchpads for Intermediate Computation with Language Models
Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.00114 · Cited by 144 · see citing works
Besta, M., Gerstenberger, R., Peter, E., et al. (2023) · ACM Computing Surveys · doi:10.1145/3604932 · Cited by 134 · see citing works
Self-citationSelection-Inference: Exploiting Large Language Models for Interpretable Logical ReasoningPreprint
Creswell, A., Shanahan, M., & Higgins, I. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.09712 · Cited by 122 · see citing works
Solving Math Word Problems via Cooperative Reasoning induced Language Models
Zhu, X., Wang, J., Zhang, L., Zhang, Y., Huang, Y.-F., Gan, R., Zhang, J., & Yang, Y. (2022) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2023.acl-long.245 · Cited by 115 · see citing works
Low-latency graph streaming using compressed purely-functional trees
Dhulipala, L., Blelloch, G. E., & Shun, J. (2019) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/3314221.3314598 · Cited by 111 · see citing works
Communication-Efficient Jaccard similarity for High-Performance Distributed Genome Comparisons
Besta, M., Kanakagiri, R., Mustafa, H., Karasikov, M., Rätsch, G., Hoefler, T., & Solomonik, E. (2020) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps47924.2020.00118 · Cited by 86 · see citing works
Self-citationSISA: Set-Centric Instruction Set Architecture for Graph Mining on Processing-in-Memory Systems
Besta, M., Kanakagiri, R., Kwaśniewski, G., et al. (2021) · doi:10.1145/3466752.3480133 · Cited by 78 · see citing works
Self-citationBeyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Large Language ModelsPreprint
Yao, Y., Li, Z., & Zhao, H. (2023) · arXiv.org · doi:10.48550/arxiv.2305.16582 · Cited by 74 · see citing works
Lifting sequential graph algorithms for distributed-memory parallel computation
Gregor, D., & Lumsdaine, A. (2005) · ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA) · doi:10.1145/1094811.1094844 · Cited by 70 · see citing works
Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data
Shum, K., Diao, S., & Zhang, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.811 · Cited by 65 · see citing works
Decomposition Enhances Reasoning via Self-Evaluation Guided DecodingPreprint
Xie, Y., Kawaguchi, K., Zhao, Y., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2305.00633 · Cited by 61 · see citing works
Foundation Models for Decision Making: Problems, Methods, and Opportunities
Yang, S., Nachum, O., Du, Y., Wei, J., Abbeel, P., & Schuurmans, D. (2023) · arXiv · doi:10.48550/arxiv.2303.04129 · Cited by 60 · see citing works
Practice of Streaming Processing of Dynamic Graphs: Concepts, Models, and Systems
Besta, M., Fischer, M., Kalavri, V., et al. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3131677 · Cited by 56 · see citing works
Self-citationWang, Z., Cai, S., Chen, G., Liu, A., Ma, X., & Liang, Y. (2023) · arXiv · doi:10.48550/arxiv.2302.01560 · Cited by 55 · see citing works
Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation
Ning, X., Lin, Z., Zhou, Z., Wang, Z., Yang, H., & Wang, Y. (2023) · arXiv · doi:10.48550/arxiv.2307.15337 · Cited by 36 · see citing works
REFINER: Reasoning Feedback on Intermediate RepresentationsPreprint
Paul, D., Ismayilzada, M., Peyrard, M., Borges, B., Bosselut, A., West, R., & Faltings, B. V. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.01904 · Cited by 35 · see citing works
Motif Prediction with Graph Neural Networks
Besta, M., Grob, R., Miglioli, C., et al. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539343 · Cited by 33 · see citing works
Self-citationDISTINGER: A distributed graph data structure for massive dynamic graph processing
Feng, G., Meng, X., & Ammar, K. (2015) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2015.7363954 · Cited by 29 · see citing works
Self-Evaluation Guided Beam Search for Reasoning
Xie, Y., Kawaguchi, K., Zhao, Y., Zhao, J. X., Kan, M.-Y., He, J., & Xie, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1802 · Cited by 27
Wu, T., Terry, M., & Cai, C. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.01691 · Cited by 26 · see citing works
Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency AnalysisPreprint
Besta, M., & Hoefler, T. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.09702 · Cited by 14 · see citing works
Self-citationBesta, M., Gerstenberger, R., Fischer, M., et al. (2023) · 2012 International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1145/3581784.3607068 · Cited by 12 · see citing works
Self-citationThe Future is Big Graphs! A Community View on Graph Processing Systems
Sakr, S., Bonifati, A., Voigt, H., et al. (2020) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000504426 · Cited by 10 · see citing works
ProbGraph: High-Performance and High-Accuracy Graph Mining with Probabilistic Set Representations
Besta, M., Miglioli, C., Labini, P. S., Tětek, J., Iff, P., Kanakagiri, R., Ashkboos, S., Janda, K., Podstawski, M., Kwaśniewski, G., et al. (2022) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41404.2022.00048 · Cited by 7 · see citing works
Self-citationAutomatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data
Shum, K., Diao, S., & Zhang, T. (2023) · arXiv · doi:10.48550/arxiv.2302.12822 · Cited by 3 · see citing works
Neural Graph DatabasesPreprint
Besta, M., Iff, P., Scheidl, F., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.09732 · Cited by 1 · see citing works
Self-citationLearning Combinatorial Node Labeling AlgorithmsPreprint
Gianinazzi, L., Fries, M., Dryden, N., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03594 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 8
- [8]8. Besta, M.; et al. 2021b. GraphMineSuite: Enabling High- Performance and Programmable Graph Mining Algorithms with Set Algebra. Proc. VLDB Endow., 14(11): 1922-1935.
- [13]13. Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; et al. 2020. Language Models are Few-Shot Learn- ers. In Advances in Neural Information Processing Systems (NeurIPS '20), volume 33, 1877-1901.
- [20]20. Dohan, D.; Xu, W.; Lewkowycz, A.; Austin, J.; Bieber, D.; et al. 2022. Language Model Cascades. In Beyond Bayes: Paths To- wards Universal Reasoning Systems, Workshop at ICML '22.
- [21]21. Fan, W.; Li, J.; Ma, S.; Tang, N.; Wu, Y.; and Wu, Y. 2010. Graph Pattern Matching: From Intractable to Polynomial Time. Proc. VLDB Endow., 3(1-2): 264-275.
- [30]30. PMLR.
- [37]37. Malewicz, G.; Austern, M. H.; Bik, A. J.; et al. 2010. Pregel: A System for Large-Scale Graph Processing. In International Con- ference on Management of Data, SIGMOD '10, 135-146. ACM.
- [51]51. Teixeira, C. H. C.; et al. 2015. Arabesque: A System for Dis- tributed Graph Mining. In 25th Symposium on Operating Systems Principles, SOSP '15, 425-440. ACM.
- [54]54. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; et al. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems (NIPS '17), volume 30.
Cited by
Sorted by citations
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941 · see citing works
Harnessing the power of llms in practice: a survey on chatGPT and beyond
Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., & Hu, X. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3649506 · Cited by 644 · see citing works
A survey on LLM-based multi-agent systems: Workflow, infrastructure, and challenges
Li, X., Wang, S., Zeng, S., Wu, Y., & Yang, Y. (2024) · Vicinagearth · doi:10.1007/s44336-024-00009-2 · Cited by 404 · see citing works
“The Graph of Thought [233] expands the tree-like reasoning structure in ToT to graph structures, resulting in more robust prompting strategies.”
Empowering biomedical discovery with AI agents
Gao, S., Fang, A., Huang, Y., Giunchiglia, V., Noori, A., Schwarz, J. R., Ektefaie, Y., Kondic, J., & Zitnik, M. (2024) · Cell · doi:10.1016/j.cell.2024.09.022 · Cited by 315 · see citing works
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274 · see citing works
Artificial intelligence for literature reviews: opportunities and challenges
Bolaños, F., Salatino, A., Osborne, F., & Motta, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10902-3 · Cited by 246 · see citing works
“These techniques include Chain-of-Thought (CoT) [118], Tree of Thoughts (ToT) [119, 120] and Graph of Thoughts (GoT) [121].”
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212 · see citing works
Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods
Cao, Y., Zhao, H., Cheng, Y., Shu, T., Chen, Y., Liu, G., Liang, G., Zhao, J., Yan, J., & Li, Y. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3497992 · Cited by 174 · see citing works
Applying large language models and chain-of-thought for automatic scoring
Lee, G.-G., Latif, E., Wu, X., Liu, N., & Zhai, X. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100213 · Cited by 186 · see citing works
Zheng, S., Pan, K., Liu, J., & Chen, Y. (2024) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2024.110382 · Cited by 127 · see citing works
Generative AI for self-adaptive systems: State of the art and research roadmap
Li, J., Zhang, M., Li, N., Weyns, D., Jin, Z., & Tei, K. (2024) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3686803 · Cited by 77 · see citing works
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
Li, H., Dong, Q., Chen, J., Su, H., Zhou, Y., Ai, Q., Ye, Z., & Liu, Y. (2024) · arXiv · doi:10.48550/arxiv.2412.05579 · Cited by 40 · see citing works
A LLM-informed multi-agent AI system for drone-based visual inspection for infrastructure
Liu, J., Li, H., Chai, C., Chen, K., & Wang, D. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103643 · Cited by 32 · see citing works
Improving Multi-Agent Debate with Sparse Communication Topology
Li, Y., Du, Y., Zhang, J., Hou, L., Grabowski, P., Li, Y., & Ie, E. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.427 · Cited by 20 · see citing works
Beyond Accuracy: Investigating Error Types in GPT-4 Responses to USMLE Questions
Roy, S., Khatua, A., Ghoochani, F., Hadler, U., Nejdl, W., & Ganguly, N. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657882 · Cited by 17 · see citing works
From Illusion to Insight: A Taxonomic Survey of Hallucination Mitigation Techniques in LLMs
Kazlaris, I., Antoniou, E., Diamantaras, K., & Bratsas, C. (2025) · doi:10.20944/preprints202508.1942.v1 · Cited by 13 · see citing works
A Survey of Context Engineering for Large Language Models
Mei, L., Yao, J., Ge, Y., Wang, Y., Bi, B., Cai, Y., Liu, J., Li, M., Li, Z.-Z., Zhang, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.13334 · Cited by 13 · see citing works
Arithmetic Reasoning with LLM: Prolog Generation & Permutation
Yang, X., Chen, B., & Tam, Y.-C. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers) · doi:10.18653/v1/2024.naacl-short.61 · Cited by 12 · see citing works
Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents
Putta, P., Mills, E., Garg, N., Motwani, S., Finn, C., Garg, D., & Rafailov, R. (2024) · arXiv · doi:10.48550/arxiv.2408.07199 · Cited by 10 · see citing works
Language-Models-as-a-Service: Overview of a New Paradigm and its Challenges
La Malfa, E., Petrov, A., Frieder, S., Weinhuber, C., Burnell, R., Nazar, R., Cohn, A., Shadbolt, N., & Wooldridge, M. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15865 · Cited by 9 · see citing works
Scaling Large Language Model-based Multi-Agent Collaboration
Qian, C., Xie, Z., Wang, Y., Liu, W., Zhu, K., Xia, H., Dang, Y., Du, Z., Chen, W., Yang, C., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.07155 · Cited by 8 · see citing works
“Along this line, multi-agent collaboration has emerged as an effective paradigm to integrate the specialities of different agents (Park et al., 2023; Zhou et al., 2023; Chen et al., 2024b; Chan et al., 2024; Chen et al., 2023; Cohen et al., 2023; Li et al., 2023b; Hua et al., 2023).”
Fang, J., Peng, Y., Zhang, X., Wang, Y., Yi, X., Zhang, G., Xu, Y., Wu, B., Liu, S., Li, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.07407 · Cited by 6 · see citing works
Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
Chen, X., Xu, J., Liang, T., He, Z., Pang, J., Yu, D., Song, L., Liu, Q., Zhou, M., Zhang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.21187 · Cited by 3 · see citing works
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
Sui, Y., Chuang, Y.-N., Wang, G., Zhang, J., Zhang, T., Yuan, J., Liu, H., Wen, A., Zhong, S., Zou, N., et al. (2025) · arXiv · Cited by 2 · see citing works
“Graph-of-Thoughts (GoT) prompting [6] extends this concept by structuring thoughts into a graph, allowing iterative refinement of individual reasoning steps.”
Feedback Loops With Language Models Drive In-Context Reward Hacking
Pan, A., Jones, E., Jagadeesan, M., & Steinhardt, J. (2024) · arXiv · doi:10.48550/arxiv.2402.06627 · Cited by 3 · see citing works
“In a similar vein, LLM evaluation can be used in place of task performance (Yao et al., 2023; Besta et al., 2023).”
MethodSelf-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
Zelikman, E., Lorch, E., Mackey, L., & Kalai, A. T. (2023) · arXiv · doi:10.48550/arxiv.2310.02304 · Cited by 4 · see citing works
“Graph of thoughts extends this, allowing other graph operations (where nodes are reasoning steps), such as aggregation (Besta et al., 2023).”
BackgroundAn Efficient Context-Dependent Memory Framework for LLM-Centric Agents
Gao, P., Zhao, J., Chen, X., & Yilin, L. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: Industry Track) · doi:10.18653/v1/2025.naacl-industry.80 · Cited by 4 · see citing works
STRICTA: Structured Reasoning in Critical Text Assessment for Peer Review and Beyond
Dycke, N., Zečević, M., Kuznetsov, I., Suess, B., Kersting, K., & Gurevych, I. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1107 · Cited by 1 · see citing works
LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space
Zhang, G., Meng, F., Wan, G., Li, Z., Wang, K., Yin, Z., Bai, L., & Yan, S. (2025) · arXiv · doi:10.48550/arxiv.2509.24771
“Other variants include hybrid approaches that integrate both strategies (Wang et al., 2024a; Besta et al., 2024), as well as internalized”
"Theater of Mind" for LLMs: A Cognitive Architecture Based on Global Workspace Theory
Shang, W. (2026) · arXiv · doi:10.48550/arxiv.2604.08206
“To overcome the strict linearity of these approaches, subsequent frameworks such as Tree of Thoughts [6] and Graph of Thoughts [7] introduced non-linear exploration of reasoning paths.”
Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systems
Feng, S., Wang, Z., Goyal, P., Wang, Y., Shi, W., Xia, H., Palangi, H., Zettlemoyer, L., Tsvetkov, Y., Lee, C.-Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.04510
“We compare our approach against chain-of-thought [95] and graph-of-thought [3] with the best initial LLM expert in Figure 11.”
Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory
Suzgun, M., Yuksekgonul, M., Bianchi, F., Jurafsky, D., & Zou, J. (2025) · arXiv · doi:10.48550/arxiv.2504.07952
The AI Hippocampus: How Far are We From Human Memory?
Jia, Z., Li, J., Kang, Y., Wang, Y., Wu, T., Wang, Q., Wang, X., Zhang, S., Shen, J., Li, Q., et al. (2026) · arXiv · doi:10.48550/arxiv.2601.09113
Ning, X., Tieu, K., Fu, D., Wei, T., Li, Z., Bei, Y., Zou, J., Ai, M., Liu, Z., Li, T.-W., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.18747
CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization
Shah, D., Badhe, S., Kathrotia, N., & Tiwari, P. (2026) · arXiv
SwarmSys: Decentralized Swarm-Inspired Agents for Scalable and Adaptive Reasoning
Li, R., Liu, H., Zhao, L., Li, Z., Li, J., Jiang, J., Xu, L., Zhao, C., Fan, M., & Liang, C. (2025) · arXiv · doi:10.48550/arxiv.2510.10047
Are Retrials All You Need? Enhancing Large Language Model Reasoning Without Verbalized Feedback
Potamitis, N., & Arora, A. (2025) · arXiv · doi:10.48550/arxiv.2504.12951
Decentralized Multi-Agent Systems with Shared Context
Mao, Y., Gu, J., Chauhan, A., Zhang, Q., Kang, H., & Mirhoseini, A. (2026) · arXiv · doi:10.48550/arxiv.2606.10662
“As a result, D E LM supports both motivating settings: in test-time scaling, useful findings, failures, and partial solutions can propagate through shared context rather than being repeatedly merged and rebroadcast by a main agent.”
Why Chain of Thought Fails in Clinical Text Understanding
Wu, J., Xie, K., Gu, B., Krüger, N., Lin, K. J., & Yang, J. (2025) · arXiv · doi:10.48550/arxiv.2509.21933
PeerCheck: Enhancing LLM-Generated Academic Reviews Towards Human-Level Quality
Chen, Z., Yang, Z., Ma, Y., Backes, M., & Zhang, Y. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1170
Gu, J., Jiang, X., Shi, Z., Tan, H., Zhai, X., Xu, C., Li, W., Shen, Y., Ma, S., Liu, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.15594
Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities
Bei, Y., Zhang, W., Wang, S., Chen, W., Zhou, S., Chen, H., Li, Y., Bu, J., Pan, S., Yu, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.18019
- Authors
- Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., & Hoefler, T.
- Affiliations
- ETH Zurich (CH)
- ORCID
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2024-03-24
- Volume
- 38
- Issue
- 16
- Pages
- 17682-17690
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W4393160302
- Semantic Scholar
- aade40af0d85b0b4fe15c97f6222d5c2e4d6d9b3
- Cited by
- 556
- References
- 68
- Citations by year
- 2026: 201 · 2025: 235 · 2024: 111 · 2023: 9
- FWCI
- 161.72
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- arXiv (Cornell University)
- Funders
- European High Performance Computing Joint Undertaking
- European Commission (101070141, 955513)
- H2020 European Research Council (101002047)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- graph-of-thought, large language models, prompting, chain-of-thought, tree-of-thought, graph reasoning, feedback loops, sorting, LLM reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
